arXiv 2402.178402024-10-06cs.CLcs.AIcs.CRcs.LG

遵循我的指令并泄露信息：从检索增强生成系统中可扩展的数据提取

Follow My Instruction and Spill the Beans: Scalable Data Extraction from Retrieval-Augmented Generation Systems

Harvard University（哈佛大学）；Carnegie Mellon University（卡内基梅隆大学）；Mohamed bin Zayed University of Artificial Intelligence（穆罕默德·本·扎耶德人工智能大学）

Zhenting Qi, Hanlin Zhang, Eric Xing, Sham Kakade, Himabindu Lakkaraju

阅读 PDF

arXiv 页面 ↗HTML 原文 ↗

AI总结：

该研究揭示了上下文检索式RAG语言模型存在数据存储泄露漏洞，攻击者可通过提示注入提取数据，漏洞随模型规模扩大而加剧，且位置偏差消除策略可有效缓解，在生产级GPTs上验证了攻击的高成功率。

摘要：

检索增强生成（Retrieval-Augmented Generation, RAG）通过在测试阶段引入外部知识来改进预训练模型，以实现定制化适配。我们研究了上下文检索式RAG语言模型（Language Models, LMs）中数据存储泄露的风险。我们表明，攻击者可以利用语言模型的指令遵循能力，通过提示注入轻易地从基于指令微调语言模型构建的RAG系统的数据存储中逐字提取文本数据。这种漏洞存在于Llama2、Mistral/Mixtral、Vicuna、SOLAR、WizardLM、Qwen1.5和Platypus2等多种现代语言模型中，并且随着模型规模扩大，可利用性会加剧。我们还研究了RAG设置对数据可提取性的多种影响，表明遵循意外指令来复述数据可能是现代语言模型未能有效利用上下文的结果，并进一步表明这种漏洞可以通过位置偏差消除策略大大缓解。将我们的研究扩展到生产级RAG模型GPTs，我们设计了一种攻击，在25个随机选择的定制GPTs上，最多2次查询就能以100%的成功率导致数据存储泄露；我们仅用GPTs自身生成的100次查询进行提示，就从一本77000词的书中以41%的比例逐字提取了文本数据，从一个1569000词的语料库中以3%的比例逐字提取了文本数据。

英文摘要：

Retrieval-Augmented Generation (RAG) improves pre-trained models by incorporating external knowledge at test time to enable customized adaptation. We study the risk of datastore leakage in Retrieval-In-Context RAG Language Models (LMs). We show that an adversary can exploit LMs' instruction-following capabilities to easily extract text data verbatim from the datastore of RAG systems built with instruction-tuned LMs via prompt injection. The vulnerability exists for a wide range of modern LMs that span Llama2, Mistral/Mixtral, Vicuna, SOLAR, WizardLM, Qwen1.5, and Platypus2, and the exploitability exacerbates as the model size scales up. We also study multiple effects of RAG setup on the extractability of data, indicating that following unexpected instructions to regurgitate data can be an outcome of failure in effectively utilizing contexts for modern LMs, and further show that such vulnerability can be greatly mitigated by position bias elimination strategies. Extending our study to production RAG models GPTs, we design an attack that can cause datastore leakage with a 100% success rate on 25 randomly selected customized GPTs with at most 2 queries, and we extract text data verbatim at a rate of 41% from a book of 77,000 words and 3% from a corpus of 1,569,000 words by prompting the GPTs with only 100 queries generated by themselves.

机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。