While Retrieval-Augmented Generation (RAG) has emerged as an effective approach for addressing the knowledge outdating problem in Large Language Models (LLMs), it still faces a critical challenge: the prevalence of outdated information in knowledge bases. Current research primarily focuses on incorporating up-to-date information, yet the impact of outdated information coexisting in retrieval sources remains inadequately addressed. To bridge this gap, we introduce HoH, the first benchmark specifically designed to evaluate the impact of outdated information on RAG. Our benchmark leverages token-level diff algorithms combined with LLM pipelines to efficiently create a large-scale QA dataset that accurately captures the evolution of temporal knowledge in real-world facts. Through comprehensive experiments, we reveal that outdated information significantly degrades RAG performance in two critical ways: (1) it substantially reduces response accuracy by distracting models from correct information, and (2) it can mislead models into generating potentially harmful outputs, even when current information is available. Current RAG approaches struggle with both retrieval and generation aspects when handling outdated information. These findings highlight the urgent need for innovative solutions to address the temporal challenges in RAG. Our code and data are available at: https://github.com/0russwest0/HoH.
Rethinking All Evidence: Enhancing Trustworthy Retrieval-Augmented Generation via Conflict-Driven Summarization
University of Chinese Academy of Sciences(中国科学院大学);Chinese Academy of Sciences(中国科学院);National University of Defense Technology(国防科技大学);Chongqing University of Technology(重庆理工大学)
Enhancing Vulnerability Reports with Automated and Augmented Description Summarization
Department of Computer Science at the University of Central Florida(佛罗里达中央大学计算机科学系);Department of Computer Science at Northeastern Illinois University(北伊利诺伊大学计算机科学系);Department of Computer Science at Qatar University(卡塔尔大学计算机科学系);Ewha Womans University(成均馆大学)
Ask in Any Modality: A Comprehensive Survey on Multimodal Retrieval-Augmented Generation
Computer Engineering Department, Sharif University of Technology(沙斐大学计算机工程系);College of Interdisciplinary Science and Technology, University of Tehran(德黑兰大学跨学科科学与技术学院);Computer Engineering Department, K.N. Toosi University of Technology(技术学院计算机工程系);Qatar Computing Research Institute(卡塔尔计算研究院)
Avoiding Knowledge Edit Skipping in Multi-hop Question Answering with Guided Decomposition
State Key Laboratory for Novel Software Technology, Nanjing University, China(新型软件技术国家重点实验室,南京大学,中国);National Institute of Healthcare Data Science, Nanjing University, China(医疗数据科学国家研究院,南京大学,中国)
Agentic Retrieval-Augmented Generation: A Survey on Agentic RAG
Cleveland State University(克利夫兰州立大学);Kent State University(肯特州立大学);Northeastern University(东北大学);Roux Institute at Northeastern University(东北大学鲁克斯研究所);University of Agder (UiA)(阿格德尔大学)
A MapReduce Approach to Effectively Utilize Long Context Information in Retrieval Augmented Language Models
Columbia University(哥伦比亚大学);Weill Cornell Medicine(威尔康奈尔医学院);National Institutes of Health(美国国立卫生研究院);Weill Cornell Medical College(威尔康奈尔医学院);University of Texas Southwestern Medical Center(德克萨斯大学西南医学中心);NYU Grossman School of Medicine(纽约大学格罗斯曼医学院)
School of Intelligence Science and Technology, Peking University(北京大学智能科学与技术学院);College of Computer Science and Technology, Zhejiang University(浙江大学计算机科学与技术学院);Ant Group(蚂蚁集团);Gaoling School of Artificial Intelligence, Renmin University of China(中国人民大学高瓴人工智能学院);Rutgers University(罗格斯大学)
Who Stole Your Data? A Method for Detecting Unauthorized RAG Theft
National Engineering Research Center for Software Engineering, Peking University(软件工程国家工程研究中心,北京大学);City University of Hong Kong(香港城市大学);Fudan University(复旦大学)
An Agile Method for Implementing Retrieval Augmented Generation Tools in Industrial SMEs
LAMIH CNRS/Université Polytechnique Hauts-de-France(LAMIH CNRS/上法兰西理工大学);CIRRELT;Polytechnique Montréal(蒙特利尔理工学院);Meedia;Arts et Métiers Institute of Technology(法国国立高等工程技术学院)
A Systematic Review of Key Retrieval-Augmented Generation (RAG) Systems: Progress, Gaps, and Future Directions
Bredesen Center for Interdisciplinary Research, University of Tennessee, Knoxville, USA(跨学科研究中心,田纳西大学,肯塔基州 Knoxville);National Center for Computational Sciences, Oak Ridge National Laboratory, Oak Ridge, USA(计算科学国家中心,橡树岭国家实验室,橡树岭);University of Tennessee-Oak Ridge Innovation Institute, University of Tennessee, Knoxville, USA(田纳西大学橡树岭创新研究所,田纳西大学,肯塔基州 Knoxville)
RadioRAG: Online Retrieval-augmented Generation for Radiology Question Answering
University Hospital RWTH Aachen(亚琛工业大学附属医院);Friedrich-Alexander-Universität Erlangen-Nürnberg(埃尔朗根-纽伦堡大学);Stanford University(斯坦福大学);German Heart Centre Munich(慕尼黑德国心脏中心);Technical University of Munich(慕尼黑工业大学);Klinikum rechts der Isar(伊萨尔河右岸医院);TUD Dresden University of Technology(德累斯顿工业大学);National Center for Tumor Diseases (NCT)(国家肿瘤疾病中心);University Hospital Heidelberg(海德堡大学附属医院)
CRUD-RAG: A Comprehensive Chinese Benchmark for Retrieval-Augmented Generation of Large Language Models
University of Science and Technology of China(中国科学技术大学);Institute for Advanced Algorithms Research (Shanghai)((上海)高级算法研究院);Renmin University of China(中国人民大学);AI Research Institute(360人工智能研究院)
RAG Safety: Exploring Knowledge Poisoning Attacks to Retrieval-Augmented Generation
School of Computer Science and Technology,Xi'an Jiaotong University(西安交通大学计算机科学与技术学院);Department of Computer Science, The University of Manchester(曼彻斯特大学计算机科学系);School of Computer Science and Engineering, Southeast University(东南大学计算机科学与工程学院);National University of Singapore(新加坡国立大学)
CUB: Benchmarking Context Utilisation Techniques for Language Models
Chalmers University of Technology(查尔姆斯理工大学);University of Gothenburg(哥德堡大学);Seoul National University(首尔国立大学);University of Copenhagen(哥本哈根大学);Ewha Womans University(成均馆大学)
Hybrid-IR: Dual-Path Hybrid Retrieval with Iterative Reasoning for Complex Medical Question Answering
College of Artificial Intelligence, Nanjing Agricultural University(南京农业大学人工智能学院);School of Computer Science and Engineering, Nanjing University of Science and Technology(南京理工大学计算机科学与工程学院)
Retrieval-Augmented Generation for Large Language Models: A Survey
Shanghai Research Institute for Intelligent Autonomous Systems, Tongji University(同济大学上海智能自主系统研究院);College of Design and Innovation, Tongji University(同济大学设计与创新学院)