arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2403.02691cs.CLcs.CR
收藏

InjecAgent:对工具集成大语言模型智能体中的间接提示注入进行基准测试

InjecAgent: Benchmarking Indirect Prompt Injections in Tool-Integrated Large Language Model Agents

University of Illinois Urbana-Champaign(伊利诺伊大学厄巴纳-香槟分校)

Qiusi Zhan, Zhixiang Liang, Zifan Ying, Daniel Kang

AI总结:

提出 InjecAgent 基准,含 1,054 个测试用例,系统评估 30 个工具集成 LLM 智能体的间接提示注入漏洞,揭示其显著安全风险。

摘要:

近期研究将 LLM 体现为智能体,使其能够访问工具、执行动作并与外部内容(例如电子邮件或网站)交互。然而,外部内容带来了间接提示注入(IPI)攻击风险:恶意指令被嵌入 LLM 处理的内容中,旨在操纵这些智能体对用户执行有害动作。鉴于此类攻击可能造成严重后果,建立基准以评估并缓解这些风险势在必行。在本工作中,我们提出 InjecAgent,一个旨在评估工具集成 LLM 智能体对 IPI 攻击脆弱性的基准。InjecAgent 包含 1,054 个测试用例,覆盖 17 种不同的用户工具和 62 种攻击者工具。我们将攻击意图分为两大主要类型:对用户的直接伤害和私有数据外泄。我们评估了 30 个不同的 LLM 智能体,结果表明智能体容易受到 IPI 攻击,其中采用 ReAct 提示的 GPT-4 有 24% 的时间易受攻击。对增强设置的进一步研究(攻击者指令通过黑客提示加以强化)显示成功率进一步上升,使采用 ReAct 提示的 GPT-4 上的攻击成功率几乎翻倍。我们的发现对 LLM 智能体的广泛部署提出了疑问。我们的基准可在 https://github.com/uiuc-kang-lab/InjecAgent 获取。

英文摘要:

Recent work has embodied LLMs as agents, allowing them to access tools, perform actions, and interact with external content (e.g., emails or websites). However, external content introduces the risk of indirect prompt injection (IPI) attacks, where malicious instructions are embedded within the content processed by LLMs, aiming to manipulate these agents into executing detrimental actions against users. Given the potentially severe consequences of such attacks, establishing benchmarks to assess and mitigate these risks is imperative. In this work, we introduce InjecAgent, a benchmark designed to assess the vulnerability of tool-integrated LLM agents to IPI attacks. InjecAgent comprises 1,054 test cases covering 17 different user tools and 62 attacker tools. We categorize attack intentions into two primary types: direct harm to users and exfiltration of private data. We evaluate 30 different LLM agents and show that agents are vulnerable to IPI attacks, with ReAct-prompted GPT-4 vulnerable to attacks 24% of the time. Further investigation into an enhanced setting, where the attacker instructions are reinforced with a hacking prompt, shows additional increases in success rates, nearly doubling the attack success rate on the ReAct-prompted GPT-4. Our findings raise questions about the widespread deployment of LLM Agents. Our benchmark is available at https://github.com/uiuc-kang-lab/InjecAgent.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑