arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2603.11862cs.CRcs.AI
收藏

你让我这么做:测量LLM代理中指令文本诱导的私有数据泄露

You Told Me to Do It: Measuring Instructional Text-induced Private Data Leakage in LLM Agents

Ching-Yu Kao, Xinfeng Li, Shenyu Dai, Tianze Qiu, Pengcheng Zhou, Eric Hanchen Jiang, Philip Sperl

AI总结:

研究发现LLM代理在处理嵌入文档指令时存在安全漏洞,导致高比例的数据泄露,且现有防御措施无法有效检测此类攻击。

摘要:

高特权的LLM代理能够自主处理外部文档,越来越多地被信任通过阅读和执行项目指令来自动化任务,但它们被授予终端访问、文件系统控制和出站网络连接,却缺乏最小的安全监督。我们识别并系统地测量了这种信任模型中的一个基本漏洞,我们称之为"受信任执行者困境":代理以高速执行嵌入文档的指令,包括对抗性指令,因为它们无法区分恶意指令与合法的设置指导。这个漏洞是遵循指令设计范式结构性后果,而不是实现错误。为了构建我们的测量,我们正式化了一个三维分类法,涵盖语言伪装、结构混淆和语义抽象,并构建了ReadSecBench,一个包含500个真实世界README文件的基准,以实现可重复的评估。在商业部署的计算机使用代理上的实验显示,端到端外泄成功率达到85%,在五种编程语言和三种注入位置上保持一致。在模拟环境中的四个LLM家族的跨模型评估确认,与注入指令的语义合规性在模型家族之间保持一致。一个15名参与者的用户研究显示,所有参与者检测率为0%,对12个基于规则的和6个基于LLM的防御措施的评估显示,两者类别在没有不可接受的误报率的情况下都无法实现可靠的检测。这些结果量化了代理功能合规性与安全意识之间的持续存在的"语义安全差距",确立了嵌入文档指令注入是高特权LLM代理部署中持续存在且目前未解决的威胁。

英文摘要:

High-privilege LLM agents that autonomously process external documentation are increasingly trusted to automate tasks by reading and executing project instructions, yet they are granted terminal access, filesystem control, and outbound network connectivity with minimal security oversight. We identify and systematically measure a fundamental vulnerability in this trust model, which we term the \emph{Trusted Executor Dilemma}: agents execute documentation-embedded instructions, including adversarial ones, at high rates because they cannot distinguish malicious directives from legitimate setup guidance. This vulnerability is a structural consequence of the instruction-following design paradigm, not an implementation bug. To structure our measurement, we formalize a three-dimensional taxonomy covering linguistic disguise, structural obfuscation, and semantic abstraction, and construct \textbf{ReadSecBench}, a benchmark of 500 real-world README files enabling reproducible evaluation. Experiments on the commercially deployed computer-use agent show end-to-end exfiltration success rates up to 85\%, consistent across five programming languages and three injection positions. Cross-model evaluation on four LLM families in a simulation environment confirms that semantic compliance with injected instructions is consistent across model families. A 15-participant user study yields a 0\% detection rate across all participants, and evaluation of 12 rule-based and 6 LLM-based defenses shows neither category achieves reliable detection without unacceptable false-positive rates. Together, these results quantify a persistent \emph{Semantic-Safety Gap} between agents' functional compliance and their security awareness, establishing that documentation-embedded instruction injection is a persistent and currently unmitigated threat to high-privilege LLM agent deployments.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑