JetSpec++：结合自精调的前瞻并行草稿生成
JetSpec++: Foresight Parallel Drafting with Self Refinements
AI总结：
JetSpec++通过算法-系统协同设计，结合块内自精调、块间前瞻草稿生成与流水线操作，实现推测解码的高效性，在保留目标模型输出分布的同时，获得最高7.45倍的自回归解码加速比。
摘要：
推测解码的有效性由两个指标决定：草稿生成的令牌接受率与生成草稿的成本。我们表明可通过算法-系统协同设计同时优化这两个目标，并提出JetSpec++，该模型保留了强大的学习型草稿模型，同时几乎完全隐藏其生成成本。JetSpec++包含三个组件：块内自精调，即训练草稿模型在一个令牌块内迭代修正自身的并行预测；块间前瞻草稿生成，即当前块验证完成前就开始下一个块的生成；流水线式草稿生成与验证，即重叠两个阶段，使草稿生成的边际 wall-clock 成本趋近于零。与最先进的推测解码方法相比，JetSpec++在GB200上单个请求的令牌生成速率最高达1361 tokens/秒，在7个基准测试中平均接受长度为8.94，同时保留目标模型的输出分布，其 wall-clock 加速比最高达自回归（AR）解码的7.45倍、DSpark的1.54倍。我们的代码和模型可在该https URL获取。
英文摘要：
Speculative decoding's effectiveness is governed by two quantities: the acceptance rate of drafted tokens and the cost of producing them. We show the two objectives can be co-optimized with algorithm-system co-design, and present JetSpec++, which retains a strong learned draft model while hiding its cost almost entirely. JetSpec++ combines three components: intra-block self refinement, which trains the draft model to iteratively correct its own parallel predictions within a block of tokens; inter-block foresight drafting, which begins the next block before the current one is verified; and pipelined drafting and verification, which overlaps the two stages so drafting's marginal wall-clock cost approaches zero. In comparison with state-of-the-art speculative decoding methods, JetSpec++ attains a wall-clock speedup of up to $7.45\times$ over autoregressive (AR) decoding and $1.54\times$ over DSpark, with up to 1,361 tokens per second for a single request on GB200, and an average acceptance length of 8.94 across seven benchmarks, while preserving the target model's output distribution. Our code and models are available at this https URL.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
