知道自己不知道：通过自我练习学习在多轮RAG中何时继续搜索
Knowing You Don't Know: Learning When to Continue Search in Multi-round RAG through Self-Practicing
AI总结：
提出SIM-RAG框架，通过自我练习生成合成数据训练轻量级评判器，在多轮检索中判断信息充分性以决定是否继续搜索，提升RAG自我意识，无需人工标注，实验验证有效。
摘要：
检索增强生成（RAG）在增强语言模型知识、减少AI生成幻觉方面展现出强大能力，推动了其广泛应用。然而，需要多轮检索的复杂任务仍然具有挑战性，早期的尝试往往过于乐观，缺乏良好的自我怀疑意识。当前的多轮RAG系统可能在已检索到足够信息时仍继续搜索，或者在信息或知识不足时给出错误答案。现有解决方案要么需要大量昂贵的人工标注过程监督数据，要么导致性能不佳。本文旨在通过引入新框架SIM-RAG来解决这些局限，明确增强RAG系统的自我意识和多轮检索能力。为训练SIM-RAG，我们首先让RAG系统自我练习多轮检索，通过添加中间内心独白推理步骤来增强现有的问答对，生成合成训练数据。对于每个问答对，系统可探索多条检索路径，若达到正确答案则标记为成功，否则标记为失败。利用这些数据，我们训练一个轻量级的信息充分性评判器（Critic）。在推理时，评判器评估RAG系统在每一轮是否已检索到足够信息，指导检索决策，并通过上下文强化学习提升系统级自我意识。在多个主流RAG基准上的实验表明，SIM-RAG是一种有效的多轮RAG解决方案。此外，该框架具有系统高效性，为RAG添加轻量级组件而无需修改现有LLM或搜索引擎，并且数据高效，消除了对昂贵的人工标注中间步骤检索过程监督数据的需求。
英文摘要：
Retrieval Augmented Generation (RAG) has shown strong capability in enhancing language models' knowledge and reducing AI generative hallucinations, driving its widespread use. However, complex tasks requiring multi-round retrieval remain challenging, and early attempts tend to be overly optimistic without a good sense of self-skepticism. Current multi-round RAG systems may continue searching even when enough information has already been retrieved, or they may provide incorrect answers without having sufficient information or knowledge. Existing solutions either require large amounts of expensive human-labeled process supervision data or lead to subpar performance. This paper aims to address these limitations by introducing a new framework, SIM-RAG, to explicitly enhance RAG systems' self-awareness and multi-round retrieval capabilities. To train SIM-RAG, we first let a RAG system self-practice multi-round retrieval, augmenting existing question-answer pairs with intermediate inner monologue reasoning steps to generate synthetic training data. For each pair, the system may explore multiple retrieval paths, which are labeled as successful if they reach the correct answer and unsuccessful otherwise. Using this data, we train a lightweight information sufficiency Critic. At inference time, the Critic evaluates whether the RAG system has retrieved sufficient information at each round, guiding retrieval decisions and improving system-level self-awareness through in-context reinforcement learning. Experiments across multiple prominent RAG benchmarks show that SIM-RAG is an effective multi-round RAG solution. Furthermore, this framework is system-efficient, adding a lightweight component to RAG without requiring modifications to existing LLMs or search engines, and data-efficient, eliminating the need for costly human-annotated mid-step retrieval process supervision data.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
