arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2501.01880cs.CL
收藏

大语言模型的长上下文与检索增强生成:一项评估与再审视

Long Context vs. RAG for LLMs: An Evaluation and Revisits

Nanyang Technological University(南洋理工大学);School of Computer Science, Fudan University(复旦大学计算机科学学院)

Xinze Li, Yixin Cao, Yubo Ma, Aixin Sun

AI总结:

本文重新评估了长上下文与检索增强生成两种策略,发现LC在问答基准上总体更优,而RAG在对话和一般查询中更具优势,并指出上下文相关性在现有研究中被忽视。

摘要:

扩展上下文窗口(即长上下文,LC)和使用检索器选择性地访问相关信息(即检索增强生成,RAG)是使大语言模型(LLM)能够纳入极长外部上下文的两种主要策略。本文重新审视了近期关于该主题的研究,突出其关键见解与分歧之处。随后,我们通过过滤掉无需外部上下文即可回答的问题、识别最有效的检索方法以及扩展数据集,提供了一个更全面的评估。我们表明,在问答基准测试中,LC 总体上优于 RAG,尤其是对于基于 Wikipedia 的问题。基于摘要的检索性能与 LC 相当,而基于分块的检索则表现落后。然而,RAG 在基于对话和一般性问题查询中具有优势。这些见解凸显了 RAG 与 LC 策略之间的权衡,为未来利用外部知识源优化大语言模型提供了指导。我们还对该主题进行了深入讨论,强调了现有研究中被忽视的上下文相关性的重要性。

英文摘要:

Extending context windows (i.e., Long Context, LC) and using retrievers to selectively access relevant information (i.e., Retrieval-Augmented Generation, RAG) are the two main strategies to enable LLMs to incorporate extremely long external contexts. This paper revisits recent studies on this topic, highlighting their key insights and discrepancies. We then provide a more comprehensive evaluation by filtering out questions answerable without external context, identifying the most effective retrieval methods, and expanding the datasets. We show that LC generally outperforms RAG in question-answering benchmarks, especially for Wikipedia-based questions. Summarization-based retrieval performs comparably to LC, while chunk-based retrieval lags behind. However, RAG has advantages in dialogue-based and general question queries. These insights underscore the trade-offs between RAG and LC strategies, offering guidance for future optimization of LLMs with external knowledge sources. We also provide an in-depth discussion on this topic, highlighting the overlooked importance of context relevance in existing studies.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑