通过后门在检索增强生成中进行数据提取攻击
Data Extraction Attacks in Retrieval-Augmented Generation via Backdoors
AI总结：
本文提出通过在微调阶段注入中毒数据对RAG植入后门，使攻击者能利用特定触发器操纵LLM泄露检索数据库文档，在Gemma-2B-IT上以5%中毒数据实现94.1%逐字提取成功率，揭示了RAG供应链的隐私风险。
摘要：
尽管取得了重大进展，大型语言模型（LLMs）在缺乏特定领域或最新知识时仍难以提供准确答案。检索增强生成（RAG）通过引入外部知识库解决了这一局限性，但也引入了新的攻击面。在本文中，我们研究了针对RAG知识库的数据提取攻击。我们表明，先前基于提示注入的提取攻击在很大程度上依赖于LLMs的指令遵循能力。因此，它们在对这类恶意提示响应较弱的模型上会失败——例如，我们的实验表明，最先进的攻击在Gemma-2B-IT上几乎为零成功率。此外，即使对于能够遵循这些指令的模型，我们发现微调可能会显著降低攻击性能。为了进一步揭示该漏洞，我们提出对RAG植入后门，即在微调阶段注入一小部分中毒数据，从而在LLM内部创建后门。当这个被攻陷的LLM集成到RAG系统中时，攻击者可以利用提示中的特定触发器来操纵LLM泄露检索数据库中的文档。通过精心设计中毒数据，我们实现了逐字和释义文档提取。例如，在Gemma-2B-IT上，我们展示了仅需5%的中毒数据，我们的方法在四个数据集上对逐字提取（ROUGE-L得分：82.1）实现了94.1%的平均成功率，对释义提取（平均ROUGE得分：66.4）实现了63.6%的成功率。这些结果强调了部署RAG系统时与供应链相关的隐私风险。
英文摘要：
Despite significant advancements, large language models (LLMs) still struggle with providing accurate answers when lacking domain-specific or up-to-date knowledge. Retrieval-Augmented Generation (RAG) addresses this limitation by incorporating external knowledge bases, but it also introduces new attack surfaces. In this paper, we investigate data extraction attacks targeting RAG's knowledge databases. We show that previous prompt injection-based extraction attacks largely rely on the instruction-following capabilities of LLMs. As a result, they fail on models that are less responsive to such malicious prompts -- for example, our experiments show that state-of-the-art attacks achieve near-zero success on Gemma-2B-IT. Moreover, even for models that can follow these instructions, we found fine-tuning may significantly reduce attack performance. To further reveal the vulnerability, we propose to backdoor RAG, where a small portion of poisoned data is injected during the fine-tuning phase to create a backdoor within the LLM. When this compromised LLM is integrated into a RAG system, attackers can exploit specific triggers in prompts to manipulate the LLM to leak documents from the retrieval database. By carefully designing the poisoned data, we achieve both verbatim and paraphrased document extraction. For example, on Gemma-2B-IT, we show that with only 5\% poisoned data, our method achieves an average success rate of 94.1\% for verbatim extraction (ROUGE-L score: 82.1) and 63.6\% for paraphrased extraction (average ROUGE score: 66.4) across four datasets. These results underscore the privacy risks associated with the supply chain when deploying RAG systems.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
