AgentPoison:通过投毒记忆或知识库对LLM智能体进行红队攻击
AgentPoison: Red-teaming LLM Agents via Poisoning Memory or Knowledge Bases
AI总结:
提出AgentPoison,首个针对通用及RAG型LLM智能体的后门攻击,通过约束优化生成触发器投毒记忆或知识库,无需训练微调,攻击成功率超80%且对良性性能影响极小。
摘要:
LLM智能体在各种应用中展现出卓越的性能,这主要归功于它们在推理、利用外部知识和工具、调用API以及执行动作与环境交互等方面的先进能力。当前的智能体通常利用记忆模块或检索增强生成(RAG)机制,从知识库中检索具有相似嵌入的过往知识和实例,以辅助任务规划与执行。然而,对未经验证的知识库的依赖引发了对其安全性和可信度的重大担忧。为了揭示此类漏洞,我们提出了一种新颖的红队攻击方法AgentPoison,这是首个针对通用型及基于RAG的LLM智能体的后门攻击,通过投毒其长期记忆或RAG知识库来实现。具体而言,我们将触发器生成过程构建为一个约束优化问题,通过将触发实例映射到独特的嵌入空间来优化后门触发器,从而确保当用户指令包含优化后的后门触发器时,恶意演示能够以高概率从被投毒的记忆或知识库中被检索出来。与此同时,不含触发器的良性指令仍将保持正常性能。与传统后门攻击不同,AgentPoison无需额外的模型训练或微调,且优化后的后门触发器展现出优越的迁移性、上下文连贯性和隐蔽性。大量实验证明了AgentPoison在攻击三类真实世界LLM智能体方面的有效性:基于RAG的自动驾驶智能体、知识密集型问答智能体和医疗保健EHRAgent。在每个智能体上,AgentPoison的平均攻击成功率均高于80%,对良性性能的影响极小(低于1%),且投毒率低于0.1%。
英文摘要:
LLM agents have demonstrated remarkable performance across various applications, primarily due to their advanced capabilities in reasoning, utilizing external knowledge and tools, calling APIs, and executing actions to interact with environments. Current agents typically utilize a memory module or a retrieval-augmented generation (RAG) mechanism, retrieving past knowledge and instances with similar embeddings from knowledge bases to inform task planning and execution. However, the reliance on unverified knowledge bases raises significant concerns about their safety and trustworthiness. To uncover such vulnerabilities, we propose a novel red teaming approach AgentPoison, the first backdoor attack targeting generic and RAG-based LLM agents by poisoning their long-term memory or RAG knowledge base. In particular, we form the trigger generation process as a constrained optimization to optimize backdoor triggers by mapping the triggered instances to a unique embedding space, so as to ensure that whenever a user instruction contains the optimized backdoor trigger, the malicious demonstrations are retrieved from the poisoned memory or knowledge base with high probability. In the meantime, benign instructions without the trigger will still maintain normal performance. Unlike conventional backdoor attacks, AgentPoison requires no additional model training or fine-tuning, and the optimized backdoor trigger exhibits superior transferability, in-context coherence, and stealthiness. Extensive experiments demonstrate AgentPoison's effectiveness in attacking three types of real-world LLM agents: RAG-based autonomous driving agent, knowledge-intensive QA agent, and healthcare EHRAgent. On each agent, AgentPoison achieves an average attack success rate higher than 80% with minimal impact on benign performance (less than 1%) with a poison rate less than 0.1%.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。