Retrieval-augmented generation (RAG) is widely utilized to incorporate external knowledge into large language models, thereby enhancing factuality and reducing hallucinations in question-answering (QA) tasks. A standard RAG pipeline consists of several components, such as query rewriting, document retrieval, document filtering, and answer generation. However, these components are typically optimized separately through supervised fine-tuning, which can lead to misalignments between the objectives of individual components and the overarching aim of generating accurate answers. Although recent efforts have explored using reinforcement learning (RL) to optimize specific RAG components, these approaches often focus on simple pipelines with only two components or do not adequately address the complex interdependencies and collaborative interactions among the modules. To overcome these limitations, we propose treating the complex RAG pipeline with multiple components as a multi-agent cooperative task, in which each component can be regarded as an RL agent. Specifically, we present MMOA-RAG, Multi-Module joint Optimization Algorithm for RAG, which employs multi-agent reinforcement learning to harmonize all agents' goals toward a unified reward, such as the F1 score of the final answer. Experiments conducted on various QA benchmarks demonstrate that MMOA-RAG effectively boost the overall performance of the pipeline and outperforms existing baselines. Furthermore, comprehensive ablation studies validate the contributions of individual components and demonstrate MMOA-RAG can be adapted to different RAG pipelines and benchmarks.
LevelRAG: Enhancing Retrieval-Augmented Generation with Multi-hop Logic Planning over Rewriting Augmented Searchers
Institute of Computing Technology, Chinese Academy of Sciences(中国科学院计算技术研究所);Chinese Academy of Sciences(中国科学院);University of Chinese Academy of Sciences(中国科学院大学);Harbin Institute of Technology (Shenzhen)(哈尔滨工业大学(深圳))
A Large Language Model-based Framework for Semi-Structured Tender Document Retrieval-Augmented Generation
School of Information Management Sun Yat-sen University Guangzhou, China;School of Information Management Sun Yat-sen University Guangzhou, China corresponding author
CausalRAG: Integrating Causal Graphs into Retrieval-Augmented Generation
Department of Computer and Data Sciences, Case Western Reserve University(计算机与数据科学系,凯斯西储大学);Department of Design and Innovation, Case Western Reserve University(设计与创新系,凯斯西储大学)
R^2AG: Incorporating Retrieval Information into Retrieval Augmented Generation
School of Computer Science, South China Normal University(华南师范大学计算机学院);The Hong Kong University of Science and Technology (Guangzhou)(香港科技大学(广州));The Hong Kong University of Science and Technology(香港科技大学)
Enhancing Retrieval-Augmented LMs with a Two-stage Consistency Learning Compressor
College of Intelligence and Computing, Tianjin University(天津大学智能与计算学部);Artificial Intelligence Laboratory, China Mobile Communication Group Tianjin Co., Ltd.(中国移动通信集团天津有限公司人工智能实验室);University College London(伦敦大学学院)
DioR: Adaptive Cognitive Detection and Contextual Retrieval Optimization for Dynamic Retrieval-Augmented Generation
Zhejiang Key Laboratory of Intelligent Education Technology and Application, Zhejiang Normal University(浙江智能教育技术与应用重点实验室,浙江师范大学);School of Computer Science and Technology, Zhejiang Normal University(浙江师范大学计算机科学与技术学院);School of Computer Science and Engineering, Southeast University(东南大学计算机科学与工程学院);Department of Computer Science and Engineering, Hong Kong University of Science and Technology(香港科技大学计算机科学与工程系);School of Computer Science and Technology, Soochow University(苏州大学计算机科学与技术学院)
RAR$^2$: Retrieval-Augmented Medical Reasoning via Thought-Driven Retrieval
Department of Computing, The Hong Kong Polytechnic University(计算机系,香港理工大学);Research Institute of Trustworthy Autonomous Systems and Department of Computer Science and Engineering, Southern University of Science and Technology(可信自主系统研究院和计算机科学与工程系,南方科技大学)
ParetoRAG: Leveraging Sentence-Context Attention for Robust and Efficient Retrieval-Augmented Generation
Institute of Information Engineering, Chinese Academy of Sciences(中国科学院信息工程研究所);School of Cybersecurity, University of Chinese Academy of Sciences(中国科学院大学网络空间安全学院);Alibaba Group(阿里巴巴集团)
RPO: Retrieval Preference Optimization for Robust Retrieval-Augmented Generation
National Engineering Research Center of Speech and Language Information Processing, University of Science and Technology of China(语音与语言信息处理国家工程研究中心,中国科学技术大学);State Key Laboratory of Cognitive Intelligence, iFLYTEK Research(认知智能国家重点实验室,iFLYTEK研究院)
GraphSearch: An Agentic Deep Searching Workflow for Graph Retrieval-Augmented Generation
IDEA Research, International Digital Economy Academy(IDEA研究机构、国际数字经济学院);Hong Kong University of Science and Technology (Guangzhou)(香港科学与技术大学(广州));DataArc Tech Ltd.(DataArc科技有限公司);Hithink RoyalFlush Information Network Co., Ltd(Hithink皇家Flush信息网络有限公司)
A Systematic Study of Retrieval Pipeline Design for Retrieval-Augmented Medical Question Answering
Department of Mechatronics & Industrial Engineering, Chittagong University of Engineering & Technology(吉大港工程与技术大学机电与工业工程系);Department of Mechanical Engineering, Chittagong University of Engineering & Technology(吉大港工程与技术大学机械工程系)