APEX-Searcher: 通过子目标细化信用分配以增强智能体检索增强生成
APEX-Searcher: Refining Credit Assignment with Subgoaling for Agentic Retrieval-Augmented Generation
AI总结：
针对复杂多跳问答中检索路径模糊和端到端强化学习奖励稀疏的问题，提出APEX-Searcher，通过分离规划与执行的信用分配（规划用RL优化、执行用SFT学习），在多个基准上取得一致提升。
摘要：
检索增强生成（RAG）将大型语言模型（LLMs）与外部知识连接起来，但单轮检索通常不足以应对复杂的多跳问题。为了增强复杂任务的搜索能力，大多数现有工作通过端到端训练将多轮迭代检索与推理过程相结合。虽然这些方法提高了问题解决性能，但它们仍然面临任务推理和模型训练方面的挑战，尤其是模糊的检索执行路径和端到端强化学习（RL）中的稀疏奖励，这可能导致不准确的检索结果和较低的性能。我们将这些失败归因于层次化的信用纠缠：单一的最终奖励同时更新规划和执行，因此模型无法清晰地区分规划错误和检索错误。我们提出APEX-Searcher，它采用了一种细化信用分配的范式：规划通过带有规划级奖励的RL进行优化，而执行则通过SFT学习。大量实验表明，在多跳RAG和任务规划基准上均取得了一致的提升。
英文摘要：
Retrieval-augmented generation (RAG) connects large language models (LLMs) to external knowledge, but single-round retrieval is often insufficient for complex multi-hop questions. To enhance search capabilities for complex tasks, most existing works integrate multi-round iterative retrieval with reasoning processes via end-to-end training. While these approaches improve problem-solving performance, they still face challenges in task reasoning and model training, especially ambiguous retrieval execution paths and sparse rewards in end-to-end reinforcement learning (RL), which can lead to inaccurate retrieval results and lower performance. We attribute these failures to hierarchical credit entanglement: a single final reward updates planning and execution together, so the model cannot clearly separate plan errors from retrieval errors. We propose APEX-Searcher, which uses a Refining Credit Assignment paradigm: planning is optimized by RL with a plan-level reward, while execution is learned by SFT. Extensive experiments show consistent gains in both multi-hop RAG and task planning across benchmarks.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
