arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2605.30686cs.CRcs.AIcs.LG
收藏

工具调用ReAct代理中深度相关的间接提示注入:注入深度、载荷框架和轮次预算敏感性

Depth-Dependent Indirect Prompt Injection in Tool-Calling ReAct Agents: Injection Depth, Payload Framing, and Turn-Budget Sensitivity

Department of Computer Science(计算机科学系);AI and Media Analysis Lab(人工智能与媒体分析实验室);Berlin, Germany(柏林,德国)

Mohammadreza Rashidi

AI总结:

通过四个对照实验(共460次试验),研究在工具调用ReAct代理中,注入深度、载荷框架和轮次预算对间接提示注入攻击成功率的影响,发现注入深度是主导变量,且仅清理第一个工具观察可捕获67%的注入成功。

摘要:

将链式推理与工具调用交错的ReAct代理越来越多地用于实际任务,如调度、文件检索和数据访问。它们的工具观察循环创建了一个直接攻击面:控制任何工具返回值的攻击者可以嵌入指令,将代理从用户目标引开,这种威胁称为间接提示注入。现有基准在固定条件下评估固定注入位置的攻击成功率(ASR),留下了三个未探索的风险维度:载荷在工具序列中出现的位置(注入深度)、使用的修辞风格(框架)以及代理允许的轮次数(轮次上限)。我们在五个攻击类别的20个场景中进行了四项对照研究,总共对GPT-4o-mini和Claude Haiku进行了460次试验,总API成本低于0.36美元。研究1显示,GPT-4o-mini的ASR从深度1的60%衰减到深度4和5的0%(Cramer's V = 0.58,p < 0.001;限制在序列内深度1-3:V = 0.47,p = 0.0013),这是由于深度1的模型抵抗和更深位置在遇到载荷前任务完成所致。研究2在Claude Haiku上重复了深度实验,通过保守的工具调用和真正的指令抵抗,在每个深度均实现了0%的ASR。研究3显示,在深度1,框架将ASR调节在25%(中性)到75%(角色)之间,范围达50个百分点,但在每个条件下N=20时未达到统计显著性。研究4确认ASR在3、5和7的轮次上限下稳定,表明轮次预算在此设置中不是风险因素。我们的结果确立了注入深度为主导变量,并表明仅清理第一个工具观察可捕获67%的测量注入成功。

英文摘要:

ReAct agents that interleave chain-of-thought reasoning with tool calls are increasingly deployed for real tasks such as scheduling, file retrieval, and data access. Their tool observation loop creates a direct attack surface: an adversary who controls any tool's return value can embed instructions that redirect the agent away from the user's goal, a threat known as indirect prompt injection. Existing benchmarks evaluate attack success rate (ASR) at a fixed injection position under fixed conditions, leaving three risk dimensions unexplored: where in the tool sequence the payload appears (injection depth), what rhetorical register it uses (framing), and how many turns the agent is permitted (turn cap). We conduct four controlled studies on 20 scenarios spanning five attack categories, totalling 460 trials against GPT-4o-mini and Claude Haiku at a combined API cost under 0.36 USD. Study 1 shows that ASR against GPT-4o-mini decays from 60% at depth 1 to 0% at depths 4 and 5 (Cramer's V = 0.58, p < 0.001; restricted to within-sequence depths 1-3: V = 0.47, p = 0.0013), driven by model resistance at depth 1 and task completion before payload encounter at deeper positions. Study 2 replicates the depth experiment on Claude Haiku, which achieves 0% ASR at every depth through a combination of conservative tool invocation and genuine instruction resistance. Study 3 shows that framing modulates ASR between 25% (neutral) and 75% (persona) at depth 1, a 50-percentage-point range that does not reach statistical significance at N = 20 per condition. Study 4 confirms that ASR is stable across turn caps of 3, 5, and 7, indicating the turn budget is not a risk factor in this setting. Our results establish injection depth as the dominant variable and show that sanitising only the first tool observation captures 67% of measured injection successes.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑