Retrieval Augmented Generation (RAG) has emerged as a crucial technique for enhancing the accuracy of Large Language Models (LLMs) by incorporating external information. With the advent of LLMs that support increasingly longer context lengths, there is a growing interest in understanding how these models perform in RAG scenarios. Can these new long context models improve RAG performance? This paper presents a comprehensive study of the impact of increased context length on RAG performance across 20 popular open source and commercial LLMs. We ran RAG workflows while varying the total context length from 2,000 to 128,000 tokens (and 2 million tokens when possible) on three domain-specific datasets, and report key insights on the benefits and limitations of long context in RAG applications. Our findings reveal that while retrieving more documents can improve performance, only a handful of the most recent state of the art LLMs can maintain consistent accuracy at long context above 64k tokens. We also identify distinct failure modes in long context scenarios, suggesting areas for future research.
Transforming External Knowledge into Triplets for Enhanced Retrieval in RAG of LLMs
School of Computing, Kyung Hee University(韩国庆熙大学计算机学院);School of Computer Science and Engineering, University of Electronic Science and Technology of China(电子科技大学计算机科学与工程学院);School of Information and Software Engineering, University of Electronic Science and Technology of China(电子科技大学信息与软件工程学院);School of Robotics and Advanced Manufacture, Harbin Institute of Technology(哈尔滨工业大学机器人与先进制造学院);School of Computer Science and Technology, Tongji University(同济大学计算机科学与技术学院)
Eliciting Critical Reasoning in Retrieval-Augmented Language Models via Contrastive Explanations
Idiap Research Institute(Idiap研究机构);Department of Computer Science, University of Manchester(曼彻斯特大学计算机科学系);National Biomarker Centre (NBC), CRUK Manchester Institute(国家生物标志物中心(NBC),CRUK曼彻斯特研究所)
Evidence-backed Fact Checking using RAG and Few-Shot In-Context Learning with LLMs
IIT Kharagpur(印度理工学院克勒格布尔分校);Aditya English Medium School(阿迪蒂亚英语中学);UCLA(加利福尼亚大学洛杉矶分校);Stanford University(斯坦福大学);University of South Carolina(南卡罗来纳大学);Amazon GenAI(亚马逊生成式AI部门)
DH-RAG: A Dynamic Historical Context-Powered Retrieval-Augmented Generation Method for Multi-Turn Dialogue
Hong Kong University of Science and Technology(香港科技大学);Huazhong University of Science and Technology(华中科技大学);University of California San Diego(加州大学圣地亚哥分校);University of Science and Technology of China(中国科学技术大学);Beihang University(北京航空航天大学)
Context-Guided Dynamic Retrieval for Improving Generation Quality in RAG Models
Cornell University(康奈尔大学);San Francisco State University(旧金山州立大学);Stevens Institute of Technology(史蒂文斯理工学院);The Chinese University of Hong Kong(香港中文大学);Santa Clara University(圣克拉拉大学)
Q-RAG: Long Context Multi-step Retrieval via Value-based Embedder Training
AXXX;Applied AI Institute(应用人工智能研究所);Research Center of the Artificial Intelligence Institute(人工智能研究所研究部);London Institute for Mathematical Sciences(伦敦数学科学研究所);Higher School of Economics(高等经济学院)
HGMEM: Hypergraph-based Working Memory to Improve Multi-step RAG for Long-Context Complex Relational Modeling
The Chinese University of Hong Kong.(香港中文大学);Pengcheng Laboratory.(鹏城实验室);WeChat AI, Tencent(微信AI,腾讯);University of Chinese Academy of Sciences.(中国科学院大学)