Retrieval Augmented Generation (RAG) is a highly effective paradigm for keeping LLM-based responses up-to-date and reducing the likelihood of hallucinations. Yet, RAG was recently shown to be quite vulnerable to corpus knowledge poisoning: an attacker injects misleading documents to the corpus to steer an LLM's output to an undesired response. We argue that the standard causal attention mechanism in LLMs enables harmful cross-document interactions, specifically in cases of attacks. Accordingly, we introduce a novel defense approach for RAG: Sparse Document Attention RAG (SDAG). This is a block-sparse attention mechanism that disallows cross-attention between retrieved documents. SDAG requires a minimal inference-time change to the attention mask. We present an empirical evaluation of LLM-based question answering (QA) with a variety of attack strategies on RAG. We show that our SDAG method substantially outperforms the standard causal attention mechanism. We further demonstrate the clear merits of integrating SDAG with state-of-the-art RAG defense methods. Specifically, the integration results in performance that is statistically significantly better than the state-of-the-art.
ProGRank: Probe-Gradient Reranking to Defend Dense-Retriever RAG from Corpus Poisoning
Chalmers University of Technology, Sweden（瑞典查尔姆斯理工大学）；University of Leeds, United Kingdom（英国利兹大学）；Carl von Ossietzky University of Oldenburg, Germany（德国奥尔登堡卡尔·冯·奥西特齐大学）
One Shot Dominance: Knowledge Poisoning Attack on Retrieval-Augmented Generation Systems
State Key Laboratory of Intelligent Game（智能游戏国家重点实验室）；Science and Technology on Integrated Information System Laboratory, Institute of Software Chinese Academy of Sciences（中国科学院软件研究所综合信息系统技术国家级重点实验室）；University of Chinese Academy of Sciences（中国科学院大学）；Nanyang Technological University（南洋理工大学）
RevPRAG: Revealing Poisoning Attacks in Retrieval-Augmented Generation through LLM Activation Analysis
School of Computer Science, Fudan University（复旦大学计算机科学学院）；Department of Computer Science, Worcester Polytechnic Institute（沃思堡理工学院计算机科学系）；Institute of Big Data, Fudan University（复旦大学大数据研究院）
Demo: Guide-RAG: Evidence-Driven Corpus Curation for Retrieval-Augmented Generation in Long COVID
Department of Computer Science University of Texas at Austin（计算机科学系得克萨斯大学奥斯汀分校）；School of Information University of Texas at Austin（信息学院得克萨斯大学奥斯汀分校）；McCombs School of Business University of Texas at Austin（麦库姆斯商学院得克萨斯大学奥斯汀分校）；Dell Medical School University of Texas at Austin（德克萨斯大学奥斯汀分校戴尔医学学院）
Knowledge Poisoning Attacks on Medical Multi-Modal Retrieval-Augmented Generation
Tsinghua University（清华大学）；Beijing University of Posts and Telecommunications（北京邮电大学）；Northwestern Polytechnical University（西北工业大学）；ETH Zurich（苏黎世联邦理工学院）
Hidden in the Metadata: Stealth Poisoning Attacks on Multimodal Retrieval-Augmented Generation
The City University of New York, CSI, Staten Island, NY 10314, USA（纽约城市大学，CSI，史泰登岛分校）；The City University of New York, Graduate Center, New York, NY 10016, USA（纽约城市大学研究生中心）
Benchmarking Retrieval-Augmented Generation for Chemistry
Siebel School of Computing and Data Science, University of Illinois Urbana-Champaign（伊利诺伊大学厄巴纳-香槟分校Siebel计算与数据科学学院）；National Library of Medicine, National Institutes of Health（美国国立卫生研究院国家医学图书馆）
Private-RAG: Answering Multiple Queries with LLMs while Keeping Your Data Private
Computer Science and Engineering University of California, San Diego（计算机科学与工程大学加州大学圣地亚哥分校）；Halıcıoğlu Data Science Institute University of California, San Diego（Halıcıoğlu数据科学研究所加州大学圣地亚哥分校）；Department of Computer Science University of California, Los Angeles（计算机科学系加州大学洛杉矶分校）
E-MIA: Exam-Style Black-Box Membership Inference Attacks against RAG Systems
College of Cyber Security, Jinan University（济南大学网络安全学院）；School of Computer Science, Shanghai Jiaotong University（上海交通大学计算机科学学院）；Institute of Artificial Intelligence, Beijing Advanced Innovation Center for Future Blockchain and Privacy Computing, Beihang University（北京航空航天大学人工智能研究院）；Zhongguancun Laboratory, Beijing（中关村实验室）
EcoSafeRAG: Efficient Security through Context Analysis in Retrieval-Augmented Generation
Institute of Information Engineering, Chinese Academy of Sciences（信息工程研究所，中国科学院）；School of Cybersecurity, University of Chinese Academy of Sciences（网络安全学院，中国科学院大学）；State Key Laboratory of Cyberspace Security Defense（网络空间安全防御国家重点实验室）
SeCon-RAG: A Two-Stage Semantic Filtering and Conflict-Free Framework for Trustworthy RAG
Institute of Software Chinese Academy of Sciences Beijing China（中国科学院软件研究所）；Key Laboratory of System Software (Chinese Academy of Sciences) and State Key Laboratory of Computer Science, Institute of Software, Chinese Academy of Sciences, Beijing, China（中国科学院系统软件重点实验室和计算机科学国家重点实验室）；University of Chinese Academy of Sciences, Beijing, China（中国科学院大学）；Northeast University China（东北大学）；Institute of Ai For industries Nanjing China（人工智能产业研究院）；Southwest University China（西南大学）；Nanyang Technological University Singapore（新加坡南洋理工大学）；Alibaba China（阿里巴巴（中国））
