你批准即执行:黑盒LLM代理的同意完整性
What You Approve Is What Executes: Consent Integrity for Black-Box LLM Agents
AI总结:
针对黑盒LLM代理的批准对话框存在摘要伪造漏洞,本文引入同意完整性概念,通过可信中介确保人类看到的动作与真实执行的动作一致,并分析了该机制的局限性与权衡。
摘要:
编码代理将重要操作置于人类参与的批准对话框之后,但对话框由代理自身叙述:人类批准代理编写的摘要。Lies-in-the-Loop (LITL) 攻击表明摘要是可伪造的,因此被攻破的代理可以显示良性的描述而执行不同的操作。本文通过将所见即所签(WYSIWYS)和可信路径属性引入代理批准通道,命名了缺失的属性——同意完整性:向人类展示的操作必须由边界处的可信中介根据真实操作渲染,而非代理的叙述,且路径不可被代理伪造,并与实际执行的操作绑定。与经典WYSIWYS相比有两个不同点:渲染者是敌手,边界真实情况是低级事件,必须在不信任代理的情况下解码。由于没有解码器是完备的,可实现的目标是相对于分析器的:分析器无法分类的内容将被标记为不可检查而非静默批准。原型实现了分析器、渲染器和执行绑定;完全中介和可信路径被指定但假定存在,未实现。在GTFOBins(一个包含1330个可信工具滥用的独立语料库)上,原型静默通过了10.0%(每个实例都通过可信工具);在tldr(28798个正常使用命令)上,标记了87.0%为不可检查。这两个独立测量界定了设计的核心张力:限制静默通过的可信列表也是导致过度提示的原因,而仅边界中介可以沿该前沿移动但无法逃脱。贡献在于属性、机制以及在该前沿上的诚实定位,而非已解决的防御。
英文摘要:
Coding agents gate consequential actions behind a human-in-the-loop approval dialog, but the dialog is narrated by the agent itself: the human approves a summary the agent writes. The Lies-in-the-Loop (LITL) attack shows that summary is forgeable, so a compromised agent can show a benign description while a different action runs. This paper names the missing property, Consent Integrity, by importing What You See Is What You Sign (WYSIWYS) and the trusted-path property into the agent approval channel: the action shown to the human must be rendered by a trusted mediator from the real action at the boundary, not the agent's narration, over a path the agent cannot spoof, and bound to the exact action that executes. Two twists distinguish it from classical WYSIWYS: the renderer is the adversary, and the boundary ground truth is a low-level event that must be decoded without trusting the agent. Since no decoder is complete, the realizable target is analyzer-relative: whatever the analyzer cannot classify is surfaced as uninspectable rather than silently approved. A prototype implements the analyzer, renderer, and bind-to-execution; total mediation and the trusted path are specified but assumed, not implemented. On GTFOBins, an independent corpus of 1330 trusted-tool abuses, the prototype silently passes 10.0% (every instance through a trusted tool); on tldr, 28,798 normal-usage commands, it marks 87.0% uninspectable. These two independent measurements bracket the design's central tension: the trust list that bounds silent passes is the same one that drives over-prompting, and a boundary-only mediator can move along that frontier but not escape it. The contribution is the property, the mechanism, and an honest position on that frontier, not a solved defense.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。