arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2601.04888cs.AI
收藏

SmartSearch: 基于过程奖励的查询优化用于搜索代理

SmartSearch: Process Reward-Guided Query Refinement for Search Agents

Renmin University of China(中国人民大学)

Tongyu Wen, Guanting Dong, Zhicheng Dou

AI总结:

SmartSearch通过过程奖励和查询优化机制提升搜索代理的查询质量与效率。

摘要:

基于大型语言模型(LLM)的搜索代理在解决知识密集型问题方面表现出色,通过整合信息检索能力。现有研究大多集中在优化搜索代理的推理范式,然而推理过程中中间搜索查询的质量仍被忽视。因此,生成的查询往往不够准确,导致意外的检索结果,最终限制了搜索代理的整体有效性。为缓解这一问题,我们引入了SmartSearch,该框架基于两个关键机制:(1)过程奖励,通过双重信用评估为每个中间搜索查询的质量提供细粒度监督。(2)查询优化,通过选择性地优化低质量搜索查询并基于这些优化重新生成后续搜索轮次来促进查询生成的优化。为了使搜索代理在过程奖励的指导下逐步内部化提升查询质量的能力,我们设计了一个三阶段课程学习框架。该框架引导代理从模仿、对齐到最终的泛化。实验结果表明,SmartSearch在现有基线上表现一致,额外的定量分析进一步证实了其在搜索效率和查询质量上的显著提升。代码可在https://github.com/MYVAE/SmartSearch上获得。

英文摘要:

Large language model (LLM)-based search agents have proven promising for addressing knowledge-intensive problems by incorporating information retrieval capabilities. Existing works largely focus on optimizing the reasoning paradigms of search agents, yet the quality of intermediate search queries during reasoning remains overlooked. As a result, the generated queries often remain inaccurate, leading to unexpected retrieval results and ultimately limiting search agents' overall effectiveness. To mitigate this issue, we introduce SmartSearch, a framework built upon two key mechanisms: (1) Process rewards, which provide fine-grained supervision for the quality of each intermediate search query through Dual-Level Credit Assessment. (2) Query refinement, which promotes the optimization of query generation by selectively refining low-quality search queries and regenerating subsequent search rounds based on these refinements. To enable the search agent to progressively internalize the ability to improve query quality under the guidance of process rewards, we design a three-stage curriculum learning framework. This framework guides the agent through a progression from imitation, to alignment, and ultimately to generalization. Experimental results show that SmartSearch consistently surpasses existing baselines, and additional quantitative analyses further confirm its significant gains in both search efficiency and query quality. The code is available at https://github.com/MYVAE/SmartSearch.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑