arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2611.05345cs.AI
收藏

对抗性推测推理提升长视野智能体性能

Adversarial Speculative Reasoning Improves Long Horizon Agents

Subconscious Systems Technologies, Inc.(潜意识系统科技公司);Massachusetts Institute of Technology(麻省理工学院);Princeton University(普林斯顿大学);Northeastern University(东北大学)

Hongyin Luo, Wei Fang, Lijie Yang, Owen Stepan, Daniel Suit, Jeff Krapf, Dana Wensberg, Jack O'Brien

AI总结:

本研究针对长视野智能体推理中推测解码的噪声问题,提出对抗性推测推理方法,通过训练草稿模型偏向对抗性偏好,提升了复杂智能体编码基准的端到端成功率。

摘要:

推测解码一直被纯粹视为一种系统优化手段,用于加速大语言模型解码且不改变模型行为,在此视角下,训练更优草稿模型的工作仅聚焦于提升接受率。我们认为该观点不适用于长视野智能体推理。通过在复杂智能体任务上对前沿开放权重模型采用不同推测解码方法进行测试,我们发现在验证窗口内数值噪声会累积,且推理过程的草稿接受结果会在30步后改变模型的输出分布,从而以随机方向显著影响长视野智能体的推理准确率。本研究中,我们通过控制此类噪声提升目标模型的推理能力。虽然我们无法改变目标模型验证的整体token分布,但可通过训练草稿模型使其更易被拒绝,从而以最高精度编码关键推理。我们在智能体轨迹中识别这些决策点,训练草稿模型使其偏向对抗性偏好。推理时,训练后的草稿模型与目标模型会在这些决策点精确分道扬镳,验证过程重启,使关键推理token从目标模型采样并在其噪声最小的条件下编码。该方法使关键推理在长视野下更易被索引且更稳定。在复杂智能体编码基准上,我们的方法在未改变目标模型或其输出分布的情况下提升了端到端成功率。我们的结果将推测解码机制从纯粹的加速器重塑为上下文精度的主动分配器,通过训练0.06%至0.28%的托管参数,提升了长视野智能体的准确率。

英文摘要:

Speculative decoding has been treated purely as a systems optimization that speeds up large language model decoding without changing the model behavior. Under this view, the growing effort to train better draft models has been focused solely on acceptance rate. We argue that this view does not hold true for long-horizon agent reasoning. Measuring frontier open-weight models on complex agentic tasks with different speculative decoding approaches, we found that numerical noise grows within verification windows, and the draft acceptance results of the thinking process can change the output distribution of the model after 30 steps, thus noticeably influencing the reasoning accuracy of long-horizon agents in random directions. In this work, we improve the reasoning capability of target models by controlling such noise. While we cannot change the overall token distribution verified by the target model, we can encode critical reasoning with highest precision by training the draft model to be rejected. We identify these points in agent traces and train the draft model to favor adversarial preference. At inference the trained draft and the target part ways precisely at these decision points, verification restarts, so that critical thinking tokens are sampled from the target model and encoded under its least noisy conditions. This approach makes critical reasoning better indexed and more durable over a long horizon. On complex agentic coding benchmarks, our approach raises end-to-end success rate with no change to the target model or its output distribution. Our results recast the speculative decoding mechanism from a pure accelerator into an active allocator for context precision that improves the accuracy of long-horizon agents by training 0.06% to 0.28% of hosted parameters.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑