通过强化学习优化RAG的检索
Optimizing Retrieval for RAG via Reinforcement Learning
AI总结：
通过强化学习优化RAG检索，提升检索性能并适应多样化环境。
摘要：
随着检索增强生成（RAG）的广泛应用，检索的作用正从为人类浏览检索信息转变为为AI推理检索上下文。这种转变创造了更加复杂的搜索环境，其中相关性难以预先定义。现有的检索器依赖于监督微调（SFT）使用人工标签或合成数据，导致相关性静态，难以适应多样的RAG环境。为了解决这一挑战，我们提出R3，一种通过强化学习（RL）优化RAG的检索框架。具体来说，我们采用一种RL训练范式，使检索器能够在给定的RAG环境中探索并自我改进，通过最小的手动实验或调整努力自动化学习过程。在广泛的任务上进行的大量实验表明，R3在原始检索器上提高了5.2%的RAG性能，并在4.9%上超越了最先进的检索器，同时在与LLM增强检索和基于后训练或指令微调LLM的RAG系统相比时实现了可比的结果。它既高效又实用，仅需4块GPU并在一天内完成训练。
英文摘要：
As retrieval-augmented generation (RAG) becomes more widespread, the role of retrieval is shifting from retrieving information for human browsing to retrieving context for AI reasoning. This shift creates more complex search environments, where relevance is difficult to pre-define. Existing retrievers rely on supervised fine-tuning (SFT) with human labels or synthetic data, resulting in static relevance that struggles to adapt to diverse RAG environments. To address this challenge, we propose R3, a Retrieval framework optimized for RAG through Reinforcement learning (RL). Specifically, we adopt an RL training paradigm that enables the retriever to explore and self-improve within given RAG environments, automating the learning process with minimal manual experimentation or tuning effort. Extensive experiments across diverse tasks demonstrate that R3 improves RAG performance by 5.2% over the original retriever and surpasses state-of-the-art retrievers by 4.9%, while achieving comparable results to LLM-augmented retrieval and RAG systems built on post-trained or instruction-tuned LLMs. It is both efficient and practical, requiring only 4 GPUs and completing training within a single day.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
