Reinforcement Fine-Tuning for History-Aware Dense Retriever in RAG
College of Computer Science and Technology, Zhejiang University, Hangzhou, China(浙江大学计算机科学与技术学院);Ningbo Global Innovation Center, Zhejiang University, Ningbo, China(浙江大学宁波全球创新中心);Department of Computer Science, National University of Singapore, Singapore(新加坡国立大学计算机科学系)
Yicheng Zhang, Zhen Qin, Zhaomin Wu, Wenqi Zhang, Shuiguang Deng
Retrieval-augmented generation (RAG) enables large language models (LLMs) to produce evidence-based responses, and its performance hinges on the matching between the retriever and LLMs. Retriever optimization has emerged as an efficient alternative to fine-tuning LLMs. However, existing solutions suffer from objective mismatch between retriever optimization and the goal of RAG pipeline. Reinforcement learning (RL) provides a promising solution to address this limitation, yet applying RL to retriever optimization introduces two fundamental challenges: 1) the deterministic retrieval is incompatible with RL formulations, and 2) state aliasing arises from query-only retrieval in multi-hop reasoning. To address these challenges, we replace deterministic retrieval with stochastic sampling and formulate RAG as a Markov decision process, making retriever optimizable by RL. Further, we incorporate retrieval history into the state at each retrieval step to mitigate state aliasing. Extensive experiments across diverse RAG pipelines, datasets, and retriever scales demonstrate consistent improvements of our approach in RAG performance.
Neurosymbolic Retrievers for Retrieval-augmented Generation
Dept. of CSEE University of Maryland Baltimore County, Maryland, USA(电子工程系大学马里兰大学巴尔的摩县);Dept. of CSEE University of Maryland Baltimore County, MD, USA(电子工程系大学马里兰大学巴尔的摩县)
RAGtifier: Evaluating RAG Generation Approaches of State-of-the-Art RAG Systems for the SIGIR LiveRAG Competition
L3S Research Center(L3S研究所以);Inria Paris-Rocquencourt(Inria巴黎-罗克琴克特研究所);Rajiv Gandhi University(拉吉夫·甘地大学);Tsinghua University(清华大学);Palmer Research Laboratories(帕勒姆研究实验室)
CDS4RAG: Cyclic Dual-Sequential Hyperparameter Optimization for RAG
School of Computer Science and Engineering, UESTC, Chengdu, China(电子科技大学计算机科学与工程学院,成都,中国);IDEAS Lab, University of Birmingham, Birmingham, UK(伯明翰大学IDEAS实验室,英国布里斯托尔)
C-3PO: Compact Plug-and-Play Proxy Optimization to Achieve Human-like Retrieval-Augmented Generation
Gaoling School of Artificial Intelligence, Renmin University of China(中国人民大学人工智能学院);Tongyi Lab(通义实验室);Soochow University(苏州大学);University of Oxford(牛津大学);Tsinghua University(清华大学)
Connect the Dots: Knowledge Graph-Guided Crawler Attack on Retrieval-Augmented Generation Systems
MOE Key Lab of HCST (PKU), School of Computer Science, Peking University(信息与电子技术教育部重点实验室(北京大学)、计算机科学学院、北京大学);Department of Computer Science, University of Illinois Urbana-Champaign(计算机科学系、伊利诺伊大学厄巴纳-香槟分校);Department of Electrical and Computer Engineering, University of Illinois Urbana-Champaign(电气与计算机工程系、伊利诺伊大学厄巴纳-香槟分校)