arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2602.01672cs.CL
收藏

面向搜索增强型大语言模型推理的自适应信息控制

Adaptive Information Control for Search-Augmented LLM Reasoning

Georgia Institute of Technology(佐治亚理工学院)

Siheng Xiong, Oguzhan Gungordu, James C. Kerce, Faramarz Fekri

AI总结:

提出基于信息效用的自适应控制框架DeepControl,通过控制检索的广度与分辨率,提升搜索增强推理的性能与训练稳定性。

摘要:

搜索增强型推理代理将多步推理与外部检索交错进行,但不受控制的检索可能引入冗余证据、使上下文饱和,并破坏强化学习(RL)的稳定性。现有的基于结果的RL方法仅提供稀疏的终端奖励,对中间信息获取决策的指导有限。我们提出DeepControl,一种基于信息效用的自适应信息控制框架,其中信息效用是检索证据边际价值的状态依赖估计。该框架沿两个维度调节信息获取:广度(即是否应继续检索)和分辨率(即应暴露多少检索细节)。它通过检索继续引导、层次化粒度控制以及退火控制强制方案来实现这些控制。这使得策略能够在训练期间内化有效的获取行为,并在测试时无需外部控制即可运行。在七个基准测试中,DeepControl在没有显式信息控制的情况下,始终优于强RL和检索基线;与Search-R1相比,在Qwen2.5-7B和Qwen2.5-3B上分别平均提高了9.4和8.6个点。额外分析显示搜索效率、训练稳定性和证据利用率均有所提升。

英文摘要:

Search-augmented reasoning agents interleave multi-step reasoning with external retrieval, but uncontrolled retrieval can introduce redundant evidence, saturate the context, and destabilize reinforcement learning (RL). Existing outcome-based RL methods provide only sparse terminal rewards, offering limited guidance for intermediate information-acquisition decisions. We propose DeepControl, an adaptive information-control framework based on information utility, a state-dependent estimate of the marginal value of retrieved evidence. The framework regulates information acquisition along two axes: extent, i.e., whether retrieval should continue, and resolution, i.e., how much retrieved detail should be exposed. It implements these controls through retrieval-continuation guidance, hierarchical granularity control, and an annealed control-forcing scheme. This enables the policy to internalize effective acquisition behavior during training and operate without external control at test time. Across seven benchmarks, DeepControl consistently outperforms strong RL and retrieval baselines without explicit information control; compared with Search-R1, it improves average performance by +9.4 and +8.6 points on Qwen2.5-7B and Qwen2.5-3B, respectively. Additional analyses show improved search effectiveness, training stability, and evidence utilization.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑