arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2408.05025cs.CRcs.AI
收藏

Rag and Roll:基于大语言模型的应用框架中间接提示操纵的端到端评估

Rag and Roll: An End-to-End Evaluation of Indirect Prompt Manipulations in LLM-based Application Frameworks

CISPA Helmholtz Center for Information Security(CISPA亥姆霍兹信息安全中心)

Gianluca De Stefano, Lea Schönherr, Giancarlo Pellegrino

AI总结:

该研究聚焦RAG系统的间接提示操纵安全问题,梳理RAG架构与攻击技术,构建Rag 'n Roll框架开展端到端评估,发现现有攻击成功率约40%,配置调整对防御作用有限。

摘要:

检索增强生成(Retrieval Augmented Generation,RAG)是一种常用于为模型赋予分布外知识的技术。该过程包括收集、索引、检索信息,并将信息提供给大语言模型(LLM)以生成响应。尽管RAG凭借其灵活性和低成本日益普及,但尚未有研究广泛探讨其安全隐患。这类系统的数据通常收集自公开来源,为攻击者通过间接提示注入操纵模型响应提供了入口。本文研究了RAG系统抵御端到端间接提示操纵的安全性。首先,我们梳理了现有RAG框架的流水线,推导得出一种原型架构并确定了关键参数。随后,我们调研了已有研究,梳理出攻击者可用于实施间接提示操纵的技术。最后,我们实现了Rag 'n Roll——一个用于测定针对端到端RAG应用的攻击有效性的框架。我们的结果表明,现有攻击大多经过优化,用于在检索阶段提升恶意文档的排名,但更高的排名并不会直接转化为可靠的攻击。在不同配置下,大多数攻击的成功率约为40%;若将模棱两可的答案(即同时包含预期良性答案的答案)视为攻击成功,成功率可升至60%。此外,在使用未优化文档的情况下,攻击者为目标查询部署两份(或更多)未优化文档,可取得与使用优化文档相近的效果。最后,对RAG配置空间的探索显示,其在阻止攻击方面的作用有限,其中防御效果最佳的配置组合会严重削弱系统功能。

英文摘要:

Retrieval Augmented Generation (RAG) is a technique commonly used to equip models with out of distribution knowledge. This process involves collecting, indexing, retrieving, and providing information to an LLM for generating responses. Despite its growing popularity due to its flexibility and low cost, the security implications of RAG have not been extensively studied. The data for such systems are often collected from public sources, providing an attacker a gateway for indirect prompt injections to manipulate the responses of the model. In this paper, we investigate the security of RAG systems against end-to-end indirect prompt manipulations. First, we review existing RAG framework pipelines, deriving a prototypical architecture and identifying critical parameters. We then examine prior works searching for techniques that attackers can use to perform indirect prompt manipulations. Finally, we implemented Rag 'n Roll, a framework to determine the effectiveness of attacks against end-to-end RAG applications. Our results show that existing attacks are mostly optimized to boost the ranking of malicious documents during the retrieval phase. However, a higher rank does not immediately translate into a reliable attack. Most attacks, against various configurations, settle around a 40% success rate, which could rise to 60% when considering ambiguous answers as successful attacks (those that include the expected benign one as well). Additionally, when using unoptimized documents, attackers deploying two of them (or more) for a target query can achieve similar results as those using optimized ones. Finally, exploration of the configuration space of a RAG showed limited impact in thwarting the attacks, where the most successful combination severely undermines functionality.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑