SAAS：面向智能体搜索中过度搜索缓解的自我感知强化学习
SAAS: Self-Aware Reinforcement Learning for Over-Search Mitigation in Agentic Search
AI总结：
提出SAAS强化学习框架，通过搜索边界建模、边界感知奖励和分阶段优化策略，使LLM智能体具备动态自我感知能力，在不降低准确率的前提下显著减少过度搜索。
摘要：
智能体搜索使LLM能够通过迭代推理和外部搜索解决复杂的多跳问题。尽管有效，但这些系统在实践中常受限于一个关键缺陷：智能体无法识别自身知识边界，在内部知识足够时盲目触发搜索，甚至在已收集足够证据时未能终止搜索。缺乏自我感知导致严重的 extbf{过度搜索}，带来大量推理延迟和过高的计算成本。为此，我们提出SAAS，一种新颖的强化学习框架，旨在培养动态自我感知能力，精确调节搜索行为而不损害准确性。SAAS引入三个关键组件：(i) 搜索边界建模机制，通过对比禁用搜索和启用搜索的轨迹，识别策略演化下的搜索边界；(ii) 边界感知奖励模块，将这种边界意识转化为轨迹级惩罚，抑制不必要和冗余的搜索；(iii) 分阶段优化策略，利用顺序课程优先考虑推理而非搜索正则化，从而避免奖励黑客。大量实验表明，SAAS在保持准确性的同时大幅减少了过度搜索。我们的代码和实现细节已在https://github.com/XMUDeepLIT/SAAS发布。
英文摘要：
Agentic search enables LLMs to solve complex multi-hop questions through iterative reasoning and external search. Despite the effectiveness, these systems often suffer from a critical limitation in practice: agents fail to recognize their own knowledge boundaries, blindly triggering searches when internal knowledge suffices and failing to terminate search even when adequate evidence has been collected. The lack of self-awareness leads to severe \textbf{over-search}, incurring substantial inference latency and prohibitive computational cost. To this end, we propose SAAS, a novel RL framework designed to cultivate dynamic self-awareness that precisely regulates search behavior without compromising accuracy. SAAS introduces three key components: (i) a search boundary modeling mechanism, which identifies the search boundary under the evolving policy by contrasting search-disabled and search-enabled rollouts; (ii) a boundary-aware reward module, which translates this boundary awareness into trajectory-level penalties, suppressing unnecessary and redundant searches; and (iii) a stage-wise optimization strategy, which leverages a sequential curriculum to prioritize reasoning over search regularization, thereby avoiding reward hacking. Extensive experiments demonstrate that SAAS substantially reduces over-search, while maintaining accuracy. Our code and implementation details are released at https://github.com/XMUDeepLIT/SAAS.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
