检索增强生成评估综述
Evaluation of Retrieval-Augmented Generation: A Survey
AI总结：
本文提出Auepora统一评估流程，系统梳理RAG系统中检索与生成组件的量化指标、数据集及基准，并指出局限与未来方向。
摘要：
检索增强生成（Retrieval-Augmented Generation，RAG）近来在自然语言处理领域受到广泛关注。大量研究与实际应用正利用其通过外部信息检索增强生成模型的能力。然而，由于RAG系统的混合结构及其对动态知识源的依赖，评估这些系统面临独特挑战。为更好地理解这些挑战，我们开展了RAG统一评估流程（A Unified Evaluation Process of RAG，Auepora），旨在全面概述RAG系统的评估与基准。具体而言，我们在现有RAG基准中检查并比较检索与生成组件的若干可量化指标，如相关性、准确性和忠实度，涵盖可能的输出与真值配对。随后，我们分析各类数据集和指标，讨论当前基准的局限性，并提出推动RAG基准领域发展的潜在方向。
英文摘要：
Retrieval-Augmented Generation (RAG) has recently gained traction in natural language processing. Numerous studies and real-world applications are leveraging its ability to enhance generative models through external information retrieval. Evaluating these RAG systems, however, poses unique challenges due to their hybrid structure and reliance on dynamic knowledge sources. To better understand these challenges, we conduct A Unified Evaluation Process of RAG (Auepora) and aim to provide a comprehensive overview of the evaluation and benchmarks of RAG systems. Specifically, we examine and compare several quantifiable metrics of the Retrieval and Generation components, such as relevance, accuracy, and faithfulness, within the current RAG benchmarks, encompassing the possible output and ground truth pairs. We then analyze the various datasets and metrics, discuss the limitations of current benchmarks, and suggest potential directions to advance the field of RAG benchmarks.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
