多跳检索增强生成的保形事实性控制
Conformal Factuality Control for Multi-Hop Retrieval-Augmented Generation
浏览论文内容
中文总结 AI 辅助
本研究验证了声明级保形事实性控制在多跳RAG中的有效性，通过分割保形过滤提升事实支持率，但保留率低，需结合弃权（不执行）解释。
中文摘要 AI 辅助
检索增强生成（RAG）可以将大型语言模型基于外部证据，但检索到的上下文并不能保证生成的声明在事实上得到支持。这个问题在多跳RAG中尤为突出，因为检索和推理通过多个依赖阶段进行。我们研究先前为RAG开发的声明级保形事实性控制在此设置中是否仍然有效。我们将分割保形声明过滤应用于多跳RAG，并使用Llama 3.1 8B和GPT-4o-mini在HotpotQA、Natural Questions和TriviaQA上评估，同时进行单跳参考实验。在所有六种多跳模型-数据集配置中，越来越严格的保形目标持续增加响应中保留声明完全受支持的比例。在95%目标下，该比率范围从95.80%到97.20%，而未过滤时为55.60%-76.03%。然而，这种改进具有很强的选择性：在95%目标下，仅4.41%-31.09%的生成声明被保留，9.70%-51.40%的响应保持非空。这些结果表明，保形事实性扩展到多跳RAG，同时表明名义可靠性必须与声明保留和弃权（不执行）结合解释。
英文摘要
Retrieval-augmented generation (RAG) can ground large language models in external evidence, but retrieved context does not guarantee that generated claims are factually supported. This problem is especially relevant in multi-hop RAG, where retrieval and reasoning proceed through multiple dependent stages. We study whether claim-level conformal factuality control, previously developed for RAG, remains effective in this setting. We apply split-conformal claim filtering to multi-hop RAG and evaluate it on HotpotQA, Natural Questions, and TriviaQA using Llama 3.1 8B and GPT-4o-mini, together with a single-hop reference experiment. Across all six multi-hop model-dataset configurations, increasingly stringent conformal targets consistently increase the fraction of responses whose retained claims are fully supported. At the 95% target, this rate ranges from 95.80% to 97.20%, compared with 55.60%-76.03% without filtering. However, the improvement is strongly selective: only 4.41%-31.09% of generated claims are retained and 9.70%-51.40% of responses remain non-empty at the 95% target. These results show that conformal factuality extends to multi-hop RAG, while demonstrating that nominal reliability must be interpreted jointly with claim retention and abstention.
发表机构
- Georgia State University（佐治亚州立大学）
机构由 AI 辅助整理，请以论文原文为准。
