在应用场景中评估RAG指标：一项实验、其发现及局限性
Evaluating RAG Metrics in Applied Contexts: An Experiment, Its Findings and Its Limitations
AI总结：
研究评估RAG指标相关性，基于商业数据问答数据集，用四个库的指标对RAG系统响应等评分，与评估者分数及标准指标比较并分析相关性，指出方法局限，为未来研究提供方向。
摘要：
本文报告了一项实证研究，评估了几个RAG指标的相关性。该实验基于人类注释者从商业数据创建的问答数据集。使用来自四个库（Ragas、DeepEval、RAGChecker、Opik）的评估指标对RAG系统生成的响应和检索跨度进行评分。将这些指标与两名评估者给出的分数以及召回率等标准指标进行比较，并进行相关性分析。最后，强调了方法的某些局限性，与文献中的方法进行比较，并提出了一些未来研究的途径。本文是一篇最初发表在法语研讨会EvalLLM（布拉班特，2026年）上的论文的英文翻译。
英文摘要：
This paper reports an empirical study evaluating the relevance of several RAG metrics. The experiment is based on a question-answering dataset created by human annotators from business data. The generated responses and retrieved spans of a RAG system are scored using evaluation metrics from four libraries (Ragas, DeepEval, RAGChecker, Opik). These metrics are compared to scores given by two evaluators, as well as to standard metrics such as recall. An analysis of correlations is conducted. Finally, we highlight certain limitations of our methodology, compare it to those used in the literature, and suggest some avenues for future research. This paper is an English translation of a paper originally published in the French-speaking workshop EvalLLM (Brabant, 2026).
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
