arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2603.16169cs.IRcs.AIcs.CL
收藏

开源重现与可解释性分析的纠正检索增强生成

Open-Source Reproduction and Explainability Analysis of Corrective Retrieval Augmented Generation

College of Engineering and Applied Science University of Cincinnati(工程与应用科学学院 奥克兰大学)

Surya Vardhan Yalavarthi

AI总结:

本文开源重现了纠正检索增强生成系统,用维基百科API替代谷歌搜索,并用Phi-3-mini-4k-instruct替代原模型,验证了性能相当,并分析了检索评估器的可解释性。

摘要:

纠正检索增强生成(CRAG)通过评估检索文档质量并触发纠正措施来提高RAG系统的鲁棒性。然而,原始实现依赖于专有组件,包括谷歌搜索API和封闭模型权重,限制了可重现性。在本文中,我们展示了CRAG的完全开源重现,用维基百科API替代专有网络搜索,并用原始LLaMA-2生成器替代Phi-3-mini-4k-instruct。我们在PopQA和ARC-Challenge上进行评估,证明我们的开源流程与原始系统具有可比性能。此外,我们贡献了CRAG基于T5的检索评估器的首次可解释性分析,使用SHAP揭示评估器主要依赖于命名实体对齐而非语义相似性。我们的分析识别了关键失败模式,包括科学问题上的领域迁移限制。所有代码和结果均可在https://github.com/suryayalavarthi/crag-reproduction获取。

英文摘要:

Corrective Retrieval Augmented Generation (CRAG) improves the robustness of RAG systems by evaluating retrieved document quality and triggering corrective actions. However, the original implementation relies on proprietary components including the Google Search API and closed model weights, limiting reproducibility. In this work, we present a fully open-source reproduction of CRAG, replacing proprietary web search with the Wikipedia API and the original LLaMA-2 generator with Phi-3-mini-4k-instruct. We evaluate on PopQA and ARC-Challenge, demonstrating that our open-source pipeline achieves comparable performance to the original system. Furthermore, we contribute the first explainability analysis of CRAG's T5-based retrieval evaluator using SHAP, revealing that the evaluator primarily relies on named entity alignment rather than semantic similarity. Our analysis identifies key failure modes including domain transfer limitations on science questions. All code and results are available at https://github.com/suryayalavarthi/crag-reproduction.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑