草稿树丢失目标质量的原因：出口引导的推测解码

中文摘要 AI 辅助

基于树的推测解码可在一次目标模型前向传播中验证多个草稿延续，但由草稿分数构建的有限树面临根本的草稿-目标不匹配问题。我们探究在固定树上，更好的精确验证能否提升接受率，以及目标反馈能否改进树本身。通过目标流视角，我们确定了规范出口定律，并证明“1加目标覆盖率”可严格约束任何精确路径验证器的期望输出块长度（含奖励 token）。所有最优验证器共享相同的出口和奖励 token 定律，这已由代表性的预草稿-跟随和顺序残差验证器实现。由此得到树出口验证（TEV），这是一种精确的层级并行过程，使用一次出口节点决策和一次奖励 token 决策。出口定律还能识别缺失的目标概率，为推理时草稿树的出口引导草稿树训练（ExitTrain）提供节点级反馈。在对话、代码和数学推理任务上的实验验证了固定树等价性：ExitTrain 将平均输出块长度提升13%，TEV 将验证器阶段延迟降低15%，相较于 DDTree 实现了14%的端到端加速。我们的结果明确了两个方向：构建更高接受率的更好草稿树，以及实现更低延迟的更直接验证。代码：this https URL。

英文摘要

Tree-based speculative decoding verifies multiple draft continuations in one target-model pass, but finite trees built from draft scores face a fundamental draft-target mismatch. We ask whether better exact verification can increase acceptance on a fixed tree and how target feedback can improve the tree itself. Through a target-flow view, we identify a canonical exit law and prove that one plus target coverage sharply bounds the expected output-block length, including the bonus token, of any exact path verifier. All optimal verifiers share the same exit and bonus-token law, already attained by representative predraw-and-follow and sequential residual verifiers. This yields Tree Exit Verification (TEV), an exact, level-parallel procedure using one exit-node decision and one bonus-token decision. The exit law also identifies missing target probability, providing node-level feedback for Exit-Guided Draft-Tree Training (ExitTrain) on inference-time draft trees. Experiments across dialogue, code, and mathematical reasoning validate fixed-tree equivalence: ExitTrain increases average output-block length by 13%, while TEV reduces verifier-stage latency by 15%, yielding a 14% end-to-end speedup over DDTree. Our results distinguish two opportunities: better draft trees for higher acceptance and more direct verification for lower latency. Code: https://github.com/hsj576/TEV.

发表机构

• Fudan University（复旦大学）
• Singapore Management University（新加坡管理大学）

机构由 AI 辅助整理，请以论文原文为准。