相同载荷,不同通道:测量使用工具的語言模型中的信任不对称性
Same Payload, Different Channel: Measuring Trust Asymmetry in Tool-Using Language Models
AI总结:
本研究提出安全不对称分数(SAS),通过匹配恶意载荷仅改变传递上下文,系统测量了语言模型在不同通道(用户消息、工具元数据、工具输出)中对对抗性内容的脆弱性差异,发现代理原生模型在工具描述通道更脆弱,而通用模型相反,且机制研究表明安全相关表示在深层网络非线性编码。
摘要:
随着语言模型承担代理角色,包括调用外部API、读取工具输出以及执行嵌入在第三方内容中的指令,其攻击面远超用户输入。模型是否以相同方式处理恶意指令(无论其来源)尚未被系统研究。我们引入了安全不对称分数(SAS),通过使用匹配的载荷对(保持恶意文本相同,仅改变传递上下文)来测量模型对对抗性内容的敏感性如何随内容出现在用户消息、工具元数据或工具输出中而变化。在6个生产级LLM和三种攻击家族上的评估发现了一致且信息丰富的不对称性:当对抗性内容通过工具描述而非用户消息传递时,代理原生模型显著更脆弱,而通用模型则相反。当相同内容通过工具输出而非描述传递时,这种不对称性进一步反转,表明模型隐含地将工具元数据视为可信指令,而将工具结果视为普通数据。对Llama 3.3 70B的机制研究表明,安全相关表示在网络的中间到深层因果存在但非线性编码,解释了线性探针为何无法检测到它。这些发现揭示了当前使用工具的模型在处理对抗性内容时存在的系统性、通道依赖的盲点。
英文摘要:
As language models take on agentic roles that call APIs, read tool outputs, and act on third-party content, their attack surface expands beyond what users type. Whether they treat a malicious instruction the same way regardless of where it arrives has not been studied systematically. We introduce the Safety Asymmetry Score (SAS), measuring how a model's susceptibility to adversarial content shifts depending on whether it arrives in the user message, tool metadata, or tool output, using matched payload pairs that hold the malicious text identical and vary only the channel. Across 10 production LLMs and three attack families, general-purpose models sharply discount instructions arriving as tool metadata relative to identical instructions in the user message, while agent-native models discount them far less. This differential survives an affordance-matched control equalizing tool availability and scoring, and a size-controlled mixed-effects analysis. Within the tool surface the pattern reverses: the same content is far more instruction-like in a tool's description than in its output, with affordances identical. Models treat tool metadata as instructions and tool results as data. In Llama 3.3-70B and GPT-OSS-120B this signal is causally present at mid-to-late depths but non-linearly encoded, so linear probes miss what activation patching recovers.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。