推理时进行推测：通过联合智能体-推测器强化学习教智能体预测其下一个工具调用

中文摘要 AI 辅助

大型语言模型智能体常常花费大量实际时间等待工具调用结果。工具调用推测可以通过预测并预执行智能体的下一个工具调用（如果预测与智能体最终的工具调用匹配）来隐藏这种延迟，但现有的推测器通常是与已部署智能体自身行为对齐不佳的单独草稿模型或缓存轨迹。我们识别出这种推测器-智能体差距，并表明目标智能体本身就是一个强大的下一个调用推测器。这指向了一种更简单的设计：在同一模型中统一智能体和推测器。本文介绍了自推测智能体，它在智能体模式下解决任务，并在推测器模式下从部分轨迹预测其下一个工具调用，完全复用前缀键值缓存。为了在不降低性能的情况下实现这种双模式智能体，我们提出了一种联合智能体-推测器强化学习方法，该方法从智能体自身的展开中得出推测目标，并交替进行智能体和推测器更新。在智能体搜索问答和对话工具使用智能体任务中，我们的方法将Qwen3-4B的平均下一个工具调用Hit@1从44.1提高到61.2，将Qwen3.5-4B的从48.9提高到66.3，同时保持智能体任务成功率。

英文摘要

Large language model agents often spend substantial wall-clock time waiting for tool call results. Tool-call speculation can hide this latency by predicting and pre-executing an agent's next tool call if the prediction matches the agent's eventual tool call, but existing speculators are typically separate draft models or cached traces that are poorly aligned with the deployed agent's own behavior. We identify this speculator-agent gap and show that the target agent itself is a strong next-call speculator. This points to a simpler design: unifying the agent and speculator within the same model. In this paper, we introduce the self-speculating agent, a single model that both solves tasks in agent mode and predicts its next tool call from partial trajectories in speculator mode, fully reusing prefix KV cache. To enable this dual-mode agent without degrading performance, we propose a joint agent-speculator reinforcement learning method, which derives speculation targets from the agent's own rollouts and alternates agent and speculator updates. Across agentic search QA and conversational tool-use agentic tasks, our method improves average next tool-call Hit@1 from 44.1 to 61.2 for Qwen3-4B and from 48.9 to 66.3 for Qwen3.5-4B, while preserving agent task success.

发表机构

• University of California, Santa Barbara（加利福尼亚大学圣巴巴拉分校）
• Linkedin Inc（领英公司）

机构由 AI 辅助整理，请以论文原文为准。