SPORK：自我推测性分叉以加速智能体大语言模型推理
SPORK: Self-Speculative Forking to Accelerate Agentic LLM Inference
AI总结：
研究LLM智能体推理中串行循环耗时问题，提出SPORK方法，利用模型自身预测提前调度推测工具调用，通过成本模型等组件优化，大幅提升推理效率且不影响任务准确性。
摘要：
LLM智能体的思维-动作-观察循环通常是串行的，等待时间占比高。推测工具执行可隐藏等待，但现有系统有局限。本文提出SPORK，一个无需训练的控制器，能提前调度推测工具调用，与剩余思维链解码重叠执行。成本模型判断推测是否划算，各组件优化相关项，在真实工具基准测试中大幅提升效率。
英文摘要：
LLM agents are becoming a common interface for research, coding, and question answering, yet their Thought-Action-Observation loop is often serial: the model reasons, emits a tool call, then idles the GPU until the result returns. This wait consumes 16-37% of wall time in our workloads and 35-61% in prior reports. Speculative tool execution can hide this wait, but existing systems need auxiliary predictors, historical traces, or static workflow graphs, leaving a gap for training-free, day-one deployment. We observe that the model can be its own predictor: a probe forked at the start of generation predicts Qwen3-32B's upcoming tool name with 74.6-99.6% accuracy across five benchmarks. We present SPORK (Self-sPeculative fORKing), a training-free controller that dispatches the speculated tool call early, overlapping its execution with the remaining chain-of-thought decode. A cost model captures when speculation breaks even, and each component improves one of its terms: a prefix-cache fork cuts probe cost, a confidence gate filters mispredictions, and partial-token accept turns rejected probes into speculative-decoding drafts. On acceptance, the tool result is ready when reasoning ends; on rejection, SPORK falls back to serial execution with no correctness penalty. On real-tool benchmarks, SPORK cuts Qwen3-32B's GAIA P95 by 18% (131.9 to 108.1 s); the mechanism holds across model sizes from 4B to 32B and across dense and mixture-of-experts models, with task accuracy within 1 pp of baseline or better wherever measured. SPORK deploys as a thin controller over standard completion APIs (no retraining, no auxiliary models, no offline traces) and is orthogonal to token-level speculative decoding. SPORK is open source at https://github.com/baihuajun24/spork.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
