arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2411.03538cs.LGcs.CL
收藏

大型语言模型的长上下文RAG性能

Long Context RAG Performance of Large Language Models

Databricks Mosaic Research(Databricks Mosaic研究机构)

Quinn Leng, Jacob Portes, Sam Havens, Matei Zaharia, Michael Carbin

AI总结:

本研究通过20个LLMs在三个数据集上的实验,系统评估了长上下文对RAG性能的影响,发现仅少数最新模型能在64k以上令牌保持准确性,并识别了失败模式。

摘要:

检索增强生成(RAG)已成为通过整合外部信息来提升大型语言模型(LLMs)准确性的关键技术。随着支持越来越长上下文长度的LLMs的出现,人们越来越关注这些模型在RAG场景中的表现。这些新的长上下文模型能否提高RAG性能?本文对20个流行的开源和商业LLMs进行了关于上下文长度增加对RAG性能影响的全面研究。我们在三个领域特定数据集上运行RAG工作流,同时将总上下文长度从2,000个令牌变化到128,000个令牌(并在可能时变化到200万个令牌),并报告了关于长上下文在RAG应用中的益处和局限性的关键见解。我们的发现表明,虽然检索更多文档可以提高性能,但只有少数最新的最先进的LLMs能够在超过64k令牌的长上下文中保持一致的准确性。我们还识别了长上下文场景中的不同失败模式,为未来研究提出了方向。

英文摘要:

Retrieval Augmented Generation (RAG) has emerged as a crucial technique for enhancing the accuracy of Large Language Models (LLMs) by incorporating external information. With the advent of LLMs that support increasingly longer context lengths, there is a growing interest in understanding how these models perform in RAG scenarios. Can these new long context models improve RAG performance? This paper presents a comprehensive study of the impact of increased context length on RAG performance across 20 popular open source and commercial LLMs. We ran RAG workflows while varying the total context length from 2,000 to 128,000 tokens (and 2 million tokens when possible) on three domain-specific datasets, and report key insights on the benefits and limitations of long context in RAG applications. Our findings reveal that while retrieving more documents can improve performance, only a handful of the most recent state of the art LLMs can maintain consistent accuracy at long context above 64k tokens. We also identify distinct failure modes in long context scenarios, suggesting areas for future research.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑