该研究聚焦RAG系统的间接提示操纵安全问题,梳理RAG架构与攻击技术,构建Rag 'n Roll框架开展端到端评估,发现现有攻击成功率约40%,配置调整对防御作用有限。
摘要:
检索增强生成(Retrieval Augmented Generation,RAG)是一种常用于为模型赋予分布外知识的技术。该过程包括收集、索引、检索信息,并将信息提供给大语言模型(LLM)以生成响应。尽管RAG凭借其灵活性和低成本日益普及,但尚未有研究广泛探讨其安全隐患。这类系统的数据通常收集自公开来源,为攻击者通过间接提示注入操纵模型响应提供了入口。本文研究了RAG系统抵御端到端间接提示操纵的安全性。首先,我们梳理了现有RAG框架的流水线,推导得出一种原型架构并确定了关键参数。随后,我们调研了已有研究,梳理出攻击者可用于实施间接提示操纵的技术。最后,我们实现了Rag 'n Roll——一个用于测定针对端到端RAG应用的攻击有效性的框架。我们的结果表明,现有攻击大多经过优化,用于在检索阶段提升恶意文档的排名,但更高的排名并不会直接转化为可靠的攻击。在不同配置下,大多数攻击的成功率约为40%;若将模棱两可的答案(即同时包含预期良性答案的答案)视为攻击成功,成功率可升至60%。此外,在使用未优化文档的情况下,攻击者为目标查询部署两份(或更多)未优化文档,可取得与使用优化文档相近的效果。最后,对RAG配置空间的探索显示,其在阻止攻击方面的作用有限,其中防御效果最佳的配置组合会严重削弱系统功能。
英文摘要:
Retrieval Augmented Generation (RAG) is a technique commonly used to equip models with out of distribution knowledge. This process involves collecting, indexing, retrieving, and providing information to an LLM for generating responses. Despite its growing popularity due to its flexibility and low cost, the security implications of RAG have not been extensively studied. The data for such systems are often collected from public sources, providing an attacker a gateway for indirect prompt injections to manipulate the responses of the model. In this paper, we investigate the security of RAG systems against end-to-end indirect prompt manipulations. First, we review existing RAG framework pipelines, deriving a prototypical architecture and identifying critical parameters. We then examine prior works searching for techniques that attackers can use to perform indirect prompt manipulations. Finally, we implemented Rag 'n Roll, a framework to determine the effectiveness of attacks against end-to-end RAG applications. Our results show that existing attacks are mostly optimized to boost the ranking of malicious documents during the retrieval phase. However, a higher rank does not immediately translate into a reliable attack. Most attacks, against various configurations, settle around a 40% success rate, which could rise to 60% when considering ambiguous answers as successful attacks (those that include the expected benign one as well). Additionally, when using unoptimized documents, attackers deploying two of them (or more) for a target query can achieve similar results as those using optimized ones. Finally, exploration of the configuration space of a RAG showed limited impact in thwarting the attacks, where the most successful combination severely undermines functionality.
Defending against Indirect Prompt Injection by Instruction Detection
Renmin University of China(中国人民大学);Peking University Shenzhen Graduate School(北京大学深圳研究生院);Wuhan University(武汉大学);University of Science and Technology of China(中国科学技术大学);Hong Kong University of Science and Technology(香港科技大学);Sony AI(索尼人工智能);Microsoft Research Asia(微软亚洲研究院)
GIP-RAG: An Evidence-Grounded Retrieval-Augmented Framework for Interpretable Gene Interaction and Pathway Impact Analysis
Kanghua Juntai Biotech Co. Ltd.(康华联合生物科技有限公司);Department of Radiation Oncology, Cancer Treatment Center, The Second Affiliated Hospital of Hainan Medical University(海南医学院第二附属医院放疗科、肿瘤治疗中心);Department of Nutrition, University of California, Davis(加州大学戴维斯分校营养系);Department of Pathology, The First Affiliated Hospital of Jinan University(暨南大学第一附属医院病理科)
Hybrid RAG-empowered Multi-modal LLM for Secure Data Management in Internet of Medical Things: A Diffusion-based Contract Approach
Guangdong University of Technology(广东工业大学);Nanjing University of Aeronautics and Astronautics(南京航空航天大学);Guangzhou University of Traditional Chinese Medicine(广州中医药大学);The Second Affiliated Hospital of Guangzhou University of Chinese Medicine(广州中医药大学第二附属医院)
C-3PO: Compact Plug-and-Play Proxy Optimization to Achieve Human-like Retrieval-Augmented Generation
Gaoling School of Artificial Intelligence, Renmin University of China(中国人民大学人工智能学院);Tongyi Lab(通义实验室);Soochow University(苏州大学);University of Oxford(牛津大学);Tsinghua University(清华大学)
Knowledge Capsules: Structured Nonparametric Memory Units for LLMs
Zhejiang Angel Medical AI Technology Co., Ltd.(浙江天使医疗人工智能科技有限公司);Miti AI Technology Co., Ltd.(米蒂人工智能科技有限公司);China-Singapore Belt and Road Joint Laboratory on Translational Infection Biology for Diagnostics and Therapies(中英一带一路联合实验室(转化感染生物学诊断与治疗));State Key Laboratory for Diagnosis and Treatment of Infectious Diseases(传染病诊断与治疗国家重点实验室);The First Affiliated Hospital(第一附属医院)
Retro*: Optimizing LLMs for Reasoning-Intensive Document Retrieval
University of Science and Technology of China(中国科学技术大学);Beijing Academy of Artificial Intelligence(北京人工智能研究院);Beijing University of Posts and Telecommunications(北京邮电大学);State Key Laboratory of Cognitive Intelligence(认知智能国家重点实验室);Hong Kong Polytechnic University(香港理工大学);Hong Kong University of Science and Technology(香港科技大学)