arXivDaily arXiv每日学术速递 周一至周五更新
北美退伍军人节(Veterans Day),12 日无更新
arXiv 2601.04795cs.AIcs.CLcs.CRcs.MA
收藏

通过工具结果解析防御间接提示注入

Defense Against Indirect Prompt Injection via Tool Result Parsing

Harbin Institute of Technology(哈尔滨工业大学)

Qiang Yu, Xinran Cheng, Chuanyi Liu

AI总结:

本文提出通过工具结果解析来防御间接提示注入,有效过滤恶意代码并降低攻击成功率。

摘要:

随着LLM代理从数字助手向自主系统和机器人中的物理控制器转变,它们面临着间接提示注入的日益加剧的威胁。通过将对抗性指令嵌入工具调用的结果中,攻击者可以劫持代理的决策过程以执行未经授权的操作。这种漏洞带来了重大风险,因为代理对物理环境的直接控制能力不断增强。现有的对抗间接提示注入(IPI)防御机制通常分为两类。第一类涉及训练专用检测模型;然而,这种方法在训练和推理过程中都需要较高的计算开销,并且需要频繁更新以跟上不断演变的攻击向量。另一种是基于提示的方法,利用LLM固有的能力通过提示工程来检测或忽略恶意指令。尽管它们具有灵活性,但大多数当前的基于提示的防御方法仍然表现出较高的攻击成功率(ASR),表明对复杂的注入攻击的鲁棒性有限。在本文中,我们提出了一种新的方法,通过工具结果解析为LLM提供精确的数据,同时有效过滤出注入的恶意代码。我们的方法在攻击下的实用性(UA)方面具有竞争力,同时保持了迄今为止最低的攻击成功率(ASR),显著优于现有方法。代码可在GitHub上获取。

英文摘要:

As LLM agents transition from digital assistants to physical controllers in autonomous systems and robotics, they face an escalating threat from indirect prompt injection. By embedding adversarial instructions into the results of tool calls, attackers can hijack the agent's decision-making process to execute unauthorized actions. This vulnerability poses a significant risk as agents gain more direct control over physical environments. Existing defense mechanisms against Indirect Prompt Injection (IPI) generally fall into two categories. The first involves training dedicated detection models; however, this approach entails high computational overhead for both training and inference, and requires frequent updates to keep pace with evolving attack vectors. Alternatively, prompt-based methods leverage the inherent capabilities of LLMs to detect or ignore malicious instructions via prompt engineering. Despite their flexibility, most current prompt-based defenses suffer from high Attack Success Rates (ASR), demonstrating limited robustness against sophisticated injection attacks. In this paper, we propose a novel method that provides LLMs with precise data via tool result parsing while effectively filtering out injected malicious code. Our approach achieves competitive Utility under Attack (UA) while maintaining the lowest Attack Success Rate (ASR) to date, significantly outperforming existing methods. Code is available at GitHub.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑