Phantom：针对检索增强语言生成的通用后门攻击
Phantom: General Backdoor Attacks on Retrieval Augmented Language Generation
AI总结：
本研究针对检索增强生成（RAG）系统的安全风险，提出名为Phantom的两阶段通用后门攻击框架，通过注入带触发机制的恶意投毒文档诱导大语言模型输出违规内容，已在多款开源、闭源大模型及NVIDIA商用RAG系统上验证有效。
摘要：
检索增强生成（Retrieval Augmented Generation，RAG）通过将现代大语言模型（Large Language Models，LLMs）的响应锚定、适配并个性化到最相关的知识源，拓展了其能力边界。它在聊天机器人应用中尤为实用，开发者无需进行成本高昂的重训练即可定制LLM输出。尽管RAG系统在各类应用中实用性显著，但也带来了新的安全风险。本研究提出一种新型攻击方式，攻击者可向RAG系统的知识库注入单个恶意文档，进而实施后门投毒攻击。我们设计了Phantom——一种针对RAG系统的通用两阶段优化框架，可生成恶意投毒文档，导致模型输出出现完整性违规。第一阶段，构建的文档仅当受害者查询中出现特定的自然触发词元序列时才会被检索到；第二阶段，通过精心设计的对抗文本进一步优化文档，诱导LLM输出达成多种对抗目标，包括拒绝回答、声誉损害、隐私侵犯和有害行为。我们在Gemma、Vicuna、Llama等多个开源LLM架构上验证了攻击效果，并表明攻击可迁移至GPT-3.5 Turbo、GPT-4等闭源模型。最后，我们在端到端黑盒生产级RAG系统——NVIDIA的“Chat with RTX”上成功验证了该攻击。
英文摘要：
Retrieval Augmented Generation (RAG) expands the capabilities of modern large language models (LLMs), by anchoring, adapting, and personalizing their responses to the most relevant knowledge sources. It is particularly useful in chatbot applications, allowing developers to customize LLM output without expensive retraining. Despite their significant utility in various applications, RAG systems present new security risks. In this work, we propose a novel attack that allows an adversary to inject a single malicious document into a RAG system's knowledge base, and mount a backdoor poisoning attack. We design Phantom, a general two-stage optimization framework against RAG systems, that crafts a malicious poisoned document leading to an integrity violation in the model's output. First, the document is constructed to be retrieved only when a specific naturally occurring trigger sequence of tokens appears in the victim's queries. Second, the document is further optimized with crafted adversarial text that induces various adversarial objectives on the LLM output, including refusal to answer, reputation damage, privacy violations, and harmful behaviors.We demonstrate our attacks on multiple open-source LLM architectures, including Gemma, Vicuna, and Llama, and show that they transfer to closed-source models such as GPT-3.5 Turbo and GPT-4. Finally, we successfully demonstrate our attack on an end-to-end black-box production RAG system: NVIDIA's "Chat with RTX''.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
