RAGGED：迈向可扩展且稳定的RAG系统的科学设计
RAGGED: Towards Informed Design of Scalable and Stable RAG Systems
AI总结：
提出RAGGED框架系统评估RAG系统，发现阅读器对噪声的鲁棒性是决定其稳定性和可扩展性的关键，为优化检索深度和模型鲁棒性提供指导。
摘要：
检索增强生成（RAG）通过集成外部知识来增强语言模型，但其有效性高度依赖于系统配置。不当的检索设置会降低性能，使RAG比闭卷生成更不可靠。在本研究中，我们提出了RAGGED，这是一个用于跨多种检索器-阅读器配置、检索深度和数据集系统评估RAG系统的框架。我们的分析表明，阅读器对噪声的鲁棒性是决定RAG稳定性和可扩展性的关键因素。部分阅读器受益于检索深度的增加，而另一些则因对干扰内容的敏感性而性能下降。通过在开放域、多跳和特定领域数据集上进行的大规模实验，我们表明检索器、重排序器和提示词会影响性能，但并未从根本上改变这些由阅读器驱动的趋势。通过提供原则性框架和新指标来评估RAG的稳定性与可扩展性，RAGGED实现了对检索增强生成系统的系统评估，为未来优化检索深度和模型鲁棒性的研究提供了指导。
英文摘要：
Retrieval-augmented generation (RAG) enhances language models by integrating external knowledge, but its effectiveness is highly dependent on system configuration. Improper retrieval settings can degrade performance, making RAG less reliable than closed-book generation. In this work, we introduce RAGGED, a framework for systematically evaluating RAG systems across diverse retriever-reader configurations, retrieval depths, and datasets. Our analysis reveals that reader robustness to noise is the key determinant of RAG stability and scalability. Some readers benefit from increased retrieval depth, while others degrade due to their sensitivity to distracting content. Through large-scale experiments on open-domain, multi-hop, and specialized-domain datasets, we show that retrievers, rerankers, and prompts influence performance but do not fundamentally alter these reader-driven trends. By providing a principled framework and new metrics to assess RAG stability and scalability, RAGGED enables systematic evaluation of retrieval-augmented generation systems, guiding future research on optimizing retrieval depth and model robustness.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
