arXiv 2601.029932026-04-21cs.CL

Stable-RAG：缓解检索排列引起的幻觉

Stable-RAG: Mitigating Retrieval-Permutation-Induced Hallucinations in Retrieval-Augmented Generation

Beijing Advanced Innovation Center for Future Blockchain and Privacy Computing（北京未来区块链与隐私计算先进创新中心）；School of Artificial Intelligence, Beihang University（北航人工智能学院）；Beijing Academy of Blockchain and Edge Computing（北京区块链与边缘计算研究院）；Institute of Computing Technology, Chinese Academy of Sciences（中国科学院计算技术研究所）

Qianchi Zhang, Hainan Zhang, Liang Pang, Hongwei Zheng, Zhiming Zheng

备注：: Accepted to ACL 2026 Main

阅读 PDF

arXiv 页面 ↗HTML 原文 ↗

AI总结：

Stable-RAG通过利用检索排列敏感性估计，缓解检索排列引起的幻觉，提升问答准确性与一致性。

摘要：

检索增强生成（RAG）已成为减少大语言模型（LLMs）事实幻觉的关键范式，但检索文档顺序对模型行为的影响尚不明确。本文实证表明，在Top-5检索设置中，即使黄金文档固定在首位，检索集的排列仍会导致LLM答案显著变化，揭示了检索排列敏感性。现有鲁棒RAG方法主要关注增强LLM对低质量检索的鲁棒性及缓解位置偏见，但均未直接解决排列敏感性问题。本文提出Stable-RAG，通过排列敏感性估计缓解排列引起的幻觉。Stable-RAG在多种检索顺序下运行生成器，聚类隐藏状态，并从聚类中心表示解码，捕捉主导推理模式。随后利用这些推理结果对幻觉输出对齐至正确答案，促使模型在文档排列下产生一致且准确的预测。实验表明，Stable-RAG在三个问答数据集上相比强基线在答案准确性、推理一致性和泛化能力上均有提升。

英文摘要：

Retrieval-Augmented Generation (RAG) has become a key paradigm for reducing factual hallucinations in Large Language Models (LLMs), yet little is known about how the order of retrieved documents affects model behavior. We empirically show that under a Top-5 retrieval setting with the gold document included, LLM answers vary substantially across permutations of the retrieved set, even when the gold document is fixed in the first position. This reveals a previously underexplored sensitivity to retrieval permutations. Although existing robust RAG methods focus primarily on enhancing LLM robustness to low-quality retrieval and mitigating positional bias to distribute attention fairly over long contexts, neither approach directly addresses permutation sensitivity. In this paper, we propose Stable-RAG, which exploits permutation sensitivity estimation to mitigate permutation-induced hallucinations. Stable-RAG runs the generator under multiple retrieval orders, clusters hidden states, and decodes from a cluster-center representation that captures the dominant reasoning pattern. It then uses these reasoning results to align hallucinated outputs toward the correct answer, encouraging the model to produce consistent and accurate predictions across document permutations. Experiments on three QA datasets show that Stable-RAG improves answer accuracy, reasoning consistency, and generalization across datasets, retrievers, and input lengths compared with strong baselines.

机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。