arXiv 2505.201282025-05-26cs.CL

迭代自我激励赋能大型语言模型作为智能体搜索器

Iterative Self-Incentivization Empowers Large Language Models as Agentic Searchers

Shandong University（山东大学）；Baidu. Inc（百度公司）；Leiden University（莱顿大学）；University of Amsterdam（阿姆斯特丹大学）

Zhengliang Shi, Lingyong Yan, Dawei Yin, Suzan Verberne, Maarten de Rijke, Zhaochun Ren

备注：: Working in process

阅读 PDF

arXiv 页面 ↗HTML 原文 ↗

AI总结：

本文提出EXSEARCH智能体搜索框架，通过广义期望最大化算法实现自我激励学习，使LLM在推理中迭代优化检索决策，在四个知识密集型基准上显著提升性能。

摘要：

大型语言模型（LLMs）已被广泛集成到信息检索中，以推进传统技术。然而，由于多跳查询的复杂性以及检索到的无关内容，有效地使LLMs在复杂任务中寻求准确知识仍然是一个挑战。为了解决这些限制，我们提出了EXSEARCH，一个智能体搜索框架，其中LLM通过自我激励的过程，在推理展开时学习检索有用信息。在每一步，LLM决定检索什么（思考），触发外部检索器（搜索），并提取细粒度证据（记录）以支持下一步推理。为了使LLM具备这种能力，EXSEARCH采用了一种广义期望最大化算法。在E步中，LLM生成多个搜索轨迹，并为每个轨迹分配一个重要性权重；M步使用重新加权的损失函数在这些轨迹上训练LLM。这创建了一个自我激励的循环，LLM迭代地从自己生成的数据中学习，逐步提升自身的搜索能力。我们进一步从理论上分析了这一训练过程，建立了收敛保证。在四个知识密集型基准上的广泛实验表明，EXSEARCH显著优于基线，例如，在精确匹配分数上提高了+7.8%。受这些有前景结果的启发，我们引入了EXSEARCH-Zoo，一个将我们的方法扩展到更广泛场景的扩展，以促进未来的工作。

英文摘要：

Large language models (LLMs) have been widely integrated into information retrieval to advance traditional techniques. However, effectively enabling LLMs to seek accurate knowledge in complex tasks remains a challenge due to the complexity of multi-hop queries as well as the irrelevant retrieved content. To address these limitations, we propose EXSEARCH, an agentic search framework, where the LLM learns to retrieve useful information as the reasoning unfolds through a self-incentivized process. At each step, the LLM decides what to retrieve (thinking), triggers an external retriever (search), and extracts fine-grained evidence (recording) to support next-step reasoning. To enable LLM with this capability, EXSEARCH adopts a Generalized Expectation-Maximization algorithm. In the E-step, the LLM generates multiple search trajectories and assigns an importance weight to each; the M-step trains the LLM on them with a re-weighted loss function. This creates a self-incentivized loop, where the LLM iteratively learns from its own generated data, progressively improving itself for search. We further theoretically analyze this training process, establishing convergence guarantees. Extensive experiments on four knowledge-intensive benchmarks show that EXSEARCH substantially outperforms baselines, e.g., +7.8% improvement on exact match score. Motivated by these promising results, we introduce EXSEARCH-Zoo, an extension that extends our method to broader scenarios, to facilitate future work.

机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。