当并行草稿器遇见并行投机解码

中文摘要 AI 辅助

DSpark风格的并行草稿器使投机解码非常高效，然而其草稿阶段在每一轮的关键路径上仍然保持串行化。并行投机解码（PSD）将草稿生成与验证重叠，但现有方法必须提前猜测接受的词缀和奖励令牌：错误的猜测会使整个批次回退到串行草稿生成。我们提出DPara，一个PSD框架，它重用有效的并行草稿器，同时保证每一轮中骨干网络与验证的重叠，从而完全消除这种概率性回退。当目标验证时，DPara的扩散骨干网络为每个接受边界预计算草稿表示，而奖励令牌未指定；一个轻量级的自回归头随后将已揭示的验证结果与匹配的预计算表示结合，几乎立即生成下一轮的草稿令牌——将主导的骨干网络前向传播与验证完全并行化，仅留下可忽略的头串行成本。在Qwen3-8B和Qwen3-14B上，跨越七个数学、编码和聊天基准的实验表明，DPara相对于自回归解码实现了平均3.21倍和3.52倍的加速，同时超越了最强的串行和并行投机解码基线。

英文摘要

DSpark-style parallel drafters have made speculative decoding highly effective, yet their draft phase remains serialized on the critical path of every round. Parallel speculative decoding (PSD) overlaps drafting with verification, yet existing methods must guess the accepted prefix and bonus token in advance: a wrong guess reverts the whole batch to serial drafting. We present DPara, a PSD framework that reuses effective parallel drafters yet guarantees backbone--verification overlap in every round, thereby eliminating this probabilistic fallback altogether. While the target verifies, DPara's diffusion backbone precomputes draft representations for every acceptance boundary with the bonus left unspecified; a lightweight autoregressive head then combines the revealed verification outcome with the matching precomputed representation to emit the next round's draft tokens almost instantly---fully parallelizing the dominant backbone forward with verification and leaving only the negligible head cost serial. Experiments on Qwen3-8B and Qwen3-14B across seven math, coding, and chat benchmarks show that DPara achieves average speedups of $3.21\times$ and $3.52\times$ over autoregressive decoding, surpassing the strongest serial and parallel speculative decoding baselines alike.

发表机构

• Nanjing University（南京大学）
• Alibaba Group（阿里巴巴集团）

机构由 AI 辅助整理，请以论文原文为准。