AdaSearch：通过强化学习平衡参数知识与搜索
AdaSearch: Balancing Parametric Knowledge and Search in Large Language Models via Reinforcement Learning
AI总结：
AdaSearch通过强化学习框架平衡参数知识与搜索，提高知识边界意识并减少不必要的搜索调用。
摘要：
为大型语言模型（LLMs）配备搜索引擎已成为构建搜索代理的有效方法。然而，过度依赖搜索会引入不必要的成本，并可能暴露于噪声或恶意内容中，而仅依赖参数知识则会增加幻觉的风险。核心挑战是开发能够适应性地平衡参数知识与外部搜索的代理，仅在必要时调用搜索。先前的工作通过塑造奖励来减少搜索的过度使用，围绕工具调用次数来塑造奖励。然而，这些惩罚需要大量的奖励工程，提供模糊的信用分配，并可能被表面减少调用的代理所利用。此外，仅通过调用次数来评估性能会混淆必要和不必要的搜索，模糊了真实适应性行为的测量。为了解决这些限制，我们首先通过基于F1的决策指标量化现有搜索代理的自我知识意识，揭示了诸如Search-R1等方法往往忽视 readily available 的参数知识。受这些发现的启发，我们提出了AdaSearch，一种简单的两阶段、结果驱动的强化学习框架，将问题解决与是否调用搜索的决策分离，并使这一决策过程显式且可解释。这种透明性对于金融和医疗问答等高风险领域至关重要，但此前的方法大多忽视了这一点。在多个模型家族和大小上进行的实验表明，AdaSearch显著提高了知识边界意识，减少了不必要的搜索调用，保持了强大的任务性能，并提供了更透明、可解释的决策行为。
英文摘要：
Equipping large language models (LLMs) with search engines via reinforcement learning (RL) promises effective search agents. However, adaptively balancing internal parametric knowledge with external search remains a challenge, as overreliance on search introduces unnecessary cost and risks exposure to noisy or malicious content, while relying solely on parametric knowledge risks hallucination. Prior efforts mitigate search overuse through tool-call reward shaping, which requires heavy reward engineering and conflates necessary and unnecessary search. To address these limitations, we revisit the evaluation of search agents through an F1-based decision metric, revealing that prior methods often overlook readily available parametric knowledge. Motivated by this, we propose AdaSearch, a simple two-stage, outcome-driven RL framework that disentangles problem-solving from the decision to search, making the decision process explicit and interpretable. Extensive experiments demonstrate that AdaSearch significantly improves search-decision quality and reduces unnecessary search calls, with only a small trade-off in QA accuracy relative to always-search.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
