Multi-Head RAG: Solving Multi-Aspect Problems with LLMs
Department of Computer Science, ETH Zurich（苏黎世联邦理工学院计算机科学系）；IDEAS Research Institute（IDEAS研究 institute）；NASK National Research Institute（国家研究 institute）
Maciej Besta, Ales Kubicek, Robert Gerstenberger, Marcin Chrapek, Roman Niggli, Patrik Okanovic, Yi Zhu, Patrick Iff, Michal Podstawski, Lucas Weitzendorf, Ming… 展开作者
Maciej Besta, Ales Kubicek, Robert Gerstenberger, Marcin Chrapek, Roman Niggli, Patrik Okanovic, Yi Zhu, Patrick Iff, Michal Podstawski, Lucas Weitzendorf, Mingyuan Chi, Joanna Gajda, Piotr Nyczyk, Jürgen Müller, Hubert Niewiadomski, Torsten Hoefler
Retrieval-Augmented Generation (RAG) improves Large Language Models (LLMs) by retrieving supporting documents into the prompt, but existing methods do not explicitly target queries that require fetching multiple documents with substantially different content. Such multi-aspect queries are challenging because relevant documents can be far apart in embedding space, making joint retrieval difficult. We introduce Multi-Head RAG (MRAG), which addresses this gap with a simple yet powerful idea: using Transformer multi-head attention activations rather than the standard decoder-layer embedding, as retrieval keys. It leverages the observation that different heads capture different semantic aspects. This yields multi-aspect embeddings for both documents and queries, improving retrieval accuracy on complex queries. We show MRAG's design advantages over 18 RAG baselines, up to 20% higher retrieval success ratios for real-world use cases, and improved downstream LLM generation. MRAG integrates seamlessly with existing RAG frameworks and benchmarks.
RAG-Star: Enhancing Deliberative Reasoning with Retrieval Augmented Verification and Refinement
Gaoling School of Artificial Intelligence, Renmin University of China（中国人民大学高瓴人工智能学院）；Wuhan University of Science and Technology（武汉科技大学）；National University of Singapore（新加坡国立大学）；Northeastern University at Qinhuangdao（东北大学秦皇岛分校）；BOSS Zhipin（BOSS直聘）
ReDeEP: Detecting Hallucination in Retrieval-Augmented Generation via Mechanistic Interpretability
Gaoling School of Artificial Intelligence, Renmin University of China（中国人民大学高瓴人工智能学院）；Kuaishou Technology Co., Ltd.（快手科技有限公司）；School of Information Technology and Management, University of International Business and Economics（对外经济贸易大学信息学院）
A Data Science Approach to Calcutta High Court Judgments: An Efficient LLM and RAG-powered Framework for Summarization and Similar Cases Retrieval
A.K.Choudhury School of IT（A.K.Choudhury信息科技学院）；University of Calcutta（加尔各答大学）；Department of Computer Science（计算机科学系）；Bangabasi Morning College（Bangabasi晨学院）
Probabilistic distances-based hallucination detection in LLMs with RAG
Markov Lab, Department of Mathematics（马尔可夫实验室，数学系）；Computer Science, Saint-Petersburg University（计算机科学，圣彼得堡大学）；AI Center, Skoltech（人工智能中心，斯克里普丘克技术学院）；SB AI Lab（SB人工智能实验室）；AI Center, Skoltech, Risk department, Sber（人工智能中心，斯克里普丘克技术学院，风险部门）
SafeRAG: Benchmarking Security in Retrieval-Augmented Generation of Large Language Model
School of Information, Renmin University of China（中国人民大学信息学院）；Institute for Advanced Algorithms Research（高级算法研究院）；School of Artificial Intelligence, Beihang University（北京航空航天大学人工智能学院）
Think Parallax: Solving Multi-Hop Problems via Multi-View Knowledge-Graph-Based Retrieval-Augmented Generation
Yangtze Delta Region Institute (Huzhou), University of Electronic Science and Technology of China（长江三角洲地区研究院（湖州），电子科学与技术大学）；School of Information and Software Engineering, University of Electronic Science and Technology of China（信息与软件工程学院，电子科学与技术大学）
Attributing Response to Context: A Jensen-Shannon Divergence Driven Mechanistic Study of Context Attribution in Retrieval-Augmented Generation
University of Aberdeen（阿伯丁大学）；Nanyang Technological University（南洋理工大学）；University College London（伦敦大学学院）；University of Colorado Anschutz Medical Campus（科罗拉多大学安舒兹医学校区）；University of Sheffield（谢菲尔德大学）
Mathematical Reasoning for Unmanned Aerial Vehicles: A RAG-Based Approach for Complex Arithmetic Reasoning
Department of Computer Science（计算机科学系）；Institute For Artificial Intelligence（人工智能研究所）；Hamm-Lippstadt University of Applied Sciences（哈姆-利普施塔特应用科学大学）；University of Stuttgart（斯图加特大学）
RELOOP: Recursive Retrieval with Multi-Hop Reasoner and Planners for Heterogeneous QA
University of New South Wales（新南威尔士大学）；The Hong Kong University of Science and Technology (Guangzhou)（香港科技大学（广州））；Mohamed Bin Zayed University of Artificial Intelligence（莫扎伊德大学人工智能学院）；Technology Innovation Institute（技术创新研究所）
Improving Retrieval-Augmented Generation in Medicine with Iterative Follow-up Questions
University of Virginia（弗吉尼亚大学）；National Library of Medicine（美国国家医学图书馆）；National Institutes of Health（美国国立卫生研究院）；University of Illinois Urbana-Champaign（伊利诺伊大学厄巴纳-香槟分校）
