评估健康领域中检索增强生成对对抗性证据的鲁棒性
Evaluating the Robustness of Retrieval-Augmented Generation to Adversarial Evidence in the Health Domain
AI总结：
该研究针对健康领域RAG系统易受检索证据中错误信息影响的漏洞，通过对照实验评估其鲁棒性，发现对抗性文档会大幅降低输出一致性，而有益证据可维持鲁棒性，为高风险领域安全RAG设计提供参考并公开了实验数据。
摘要：
检索增强生成（Retrieval Augmented Generation，RAG）系统通过提供检索到的证据或上下文作为支撑，为大语言模型（Large Language Model，LLM）的回复提供事实依据的方法。在该上下文的引导下，RAG系统能够减少幻觉，并拓展LLM准确回答其训练数据范围之外问题的能力。遗憾的是，这种设计引入了一个关键漏洞：LLM可能会吸收并复现检索证据中存在的错误信息。如果检索到的证据包含明确旨在传播错误信息的对抗性内容，这一问题会被放大。\n本文对健康领域的RAG鲁棒性开展了系统性评估，并检验了模型输出与真实答案之间的一致性。我们选择聚焦健康领域，原因在于错误回复可能造成危害，同时许多常见健康相关问题都有基于循证依据的真实答案可供使用。我们利用常见健康问题开展了对照实验，分别改变检索文档的类型与构成（有益的、有害的、对抗性的）以及用户的提问框架（一致的、中立的、不一致的）。\n我们的研究结果显示，对抗性文档会大幅降低输出与真实答案的一致性，但当检索池中同时存在有益证据时，鲁棒性能够得以保持。这些发现凸显了检索防护机制的必要性，为在高风险领域设计更安全的RAG系统提供了可落地的洞见。为保障可复现性并推动未来研究，所有实验结果已在我们的GitHub仓库公开，地址为https://github.com/shakibaam/RAG_ROBUSTNESS_EVAL。
英文摘要：
Retrieval augmented generation (RAG) systems provide a method for factually grounding the responses of a Large Language Model (LLM) by providing retrieved evidence, or context, as support. Guided by this context, RAG systems can reduce hallucinations and expand the ability of LLMs to accurately answer questions outside the scope of their training data. Unfortunately, this design introduces a critical vulnerability: LLMs may absorb and reproduce misinformation present in retrieved evidence. This problem is magnified if retrieved evidence contains adversarial material explicitly intended to promulgate misinformation. This paper presents a systematic evaluation of RAG robustness in the health domain and examines alignment between model outputs and ground-truth answers. We focus on the health domain due to the potential for harm caused by incorrect responses, as well as the availability of evidence-based ground truth for many common health-related questions. We conduct controlled experiments using common health questions, varying both the type and composition of the retrieved documents (helpful, harmful, and adversarial) as well as the framing of the question by the user (consistent, neutral, and inconsistent). Our findings reveal that adversarial documents substantially degrade alignment, but robustness can be preserved when helpful evidence is also present in the retrieval pool. These findings offer actionable insights for designing safer RAG systems in high-stakes domains by highlighting the need for retrieval safeguards. To enable reproducibility and facilitate future research, all experimental results are publicly available in our github repository. https://github.com/shakibaam/RAG_ROBUSTNESS_EVAL
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
