arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2604.02640cs.CL
收藏

克服RAG的“不实际性”:提出一个现实世界基准和多维诊断框架

Overcoming the "Impracticality" of RAG: Proposing a Real-World Benchmark and Multi-Dimensional Diagnostic Framework

Kenichirou Narita, Siqi Peng, Taku Fukui, Moyuru Yamada, Satoshi Munakata, Satoru Takahashi

AI总结:

本文提出一个现实世界RAG基准和多维诊断框架,以解决企业环境中RAG系统评估的复杂挑战,提升系统可靠性。

摘要:

在企业环境中评估检索增强生成(RAG)系统性能受多维和复合因素影响,远超简单的最终准确性检查。这些因素包括推理复杂性、检索难度、文档结构多样性以及对操作可解释性的严格要求。现有学术基准未能系统诊断这些相互关联的挑战,导致高性能模型在实际部署中无法满足预期可靠性。为此,本文提出一个多维诊断框架,通过定义四轴难度分类法,并将其整合到企业RAG基准中,以诊断潜在系统弱点。

英文摘要:

Performance evaluation of Retrieval-Augmented Generation (RAG) systems within enterprise environments is governed by multi-dimensional and composite factors extending far beyond simple final accuracy checks. These factors include reasoning complexity, retrieval difficulty, the diverse structure of documents, and stringent requirements for operational explainability. Existing academic benchmarks fail to systematically diagnose these interlocking challenges, resulting in a critical gap where models achieving high performance scores fail to meet the expected reliability in practical deployment. To bridge this discrepancy, this research proposes a multi-dimensional diagnostic framework by defining a four-axis difficulty taxonomy and integrating it into an enterprise RAG benchmark to diagnose potential system weaknesses.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑