比零样本更差？一个评估RAG在误导检索中鲁棒性的事实核查数据集
Worse than Zero-shot? A Fact-Checking Dataset for Evaluating the Robustness of RAG Against Misleading Retrievals
AI总结：
RAGuard是首个评估RAG系统在误导检索中鲁棒性的基准，揭示LLM在面对误导信息时的表现劣于零样本基线。
摘要：
检索增强生成（RAG）在减轻大型语言模型（LLMs）幻觉方面展现了显著的能力。然而，当LLMs暴露于误导性或矛盾的证据时，它们在现实世界领域（如政治领域）中难以保持一致的推理，因为信息往往被极化或选择性框架。主流RAG基准测试在清洁的检索设置下评估模型，其中系统从标准文档中生成答案，或在合成扰动设置下，文档被人为注入噪声。这些假设无法反映现实条件，通常导致RAG系统性能的高估。为了解决这一差距，我们引入了RAGuard，这是第一个评估RAG系统在误导检索中鲁棒性的基准。与以往依赖合成噪声的基准不同，我们的事实核查数据集通过从Reddit讨论中构建检索语料库来捕捉自然发生的错误信息。它将检索到的证据分为三种类型：支持、误导和无关，为评估RAG系统如何处理不同类型证据提供了现实且具有挑战性的测试环境。我们的实验表明，当暴露于潜在误导性检索时，所有测试的LLM驱动RAG系统的表现都比其零样本基线（即没有检索）更差，而人类标注者始终表现更好，突显了LLM在嘈杂环境中的易受影响性。据我们所知，RAGuard是第一个系统评估RAG在误导证据下鲁棒性的基准。我们期望这个基准将推动未来研究朝着改进RAG系统的方向发展，使其在理想化数据集之外更加可靠，适用于现实应用。数据集可在https://huggingface.co/datasets/UCSC-IRKM/RAGuard获取。
英文摘要：
Retrieval-augmented generation (RAG) has shown impressive capabilities in mitigating hallucinations in large language models (LLMs). However, LLMs struggle to maintain consistent reasoning when exposed to misleading or conflicting evidence, especially in real-world domains such as politics, where information is polarized or selectively framed. Mainstream RAG benchmarks evaluate models under clean retrieval settings, where systems generate answers from gold-standard documents, or under synthetically perturbed settings, where documents are artificially injected with noise. These assumptions fail to reflect real-world conditions, often leading to an overestimation of RAG system performance. To address this gap, we introduce RAGuard, the first benchmark to evaluate the robustness of RAG systems against misleading retrievals. Unlike prior benchmarks that rely on synthetic noise, our fact-checking dataset captures naturally occurring misinformation by constructing its retrieval corpus from Reddit discussions. It categorizes retrieved evidence into three types: supporting, misleading, and unrelated, providing a realistic and challenging testbed for assessing how well RAG systems navigate different types of evidence. Our experiments reveal that, when exposed to potentially misleading retrievals, all tested LLM-powered RAG systems perform worse than their zero-shot baselines (i.e., no retrieval at all), while human annotators consistently perform better, highlighting LLMs' susceptibility to noisy environments. To our knowledge, RAGuard is the first benchmark to systematically assess the robustness of the RAG against misleading evidence. We expect this benchmark to drive future research toward improving RAG systems beyond idealized datasets, making them more reliable for real-world applications. The dataset is available at https://huggingface.co/datasets/UCSC-IRKM/RAGuard.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
