arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2610.23935cs.LGcs.AI
收藏

因子引导的扩散草稿推测解码训练与推理

Factorization-Guided Training and Inference for Diffusion-Drafted Speculative Decoding

Carnegie Mellon University(卡内基梅隆大学)

Jiakun Fan, Min Xiao, Qiang Wang, Neiwen Ling, Zhichao Liu, Zhihao Jia, Dimitrios S. Nikolopoulos, Zijun Wei

AI总结:

提出DScout方法,通过因子分解引导扩散草稿模型的训练与推理,在六个基准测试中使Qwen3-4B、Qwen3-8B的推理加速显著提升,且推理配置策略可进一步增加加速效果。

摘要:

推测解码通过并行验证草稿模型提出的token来加速大语言模型推理。扩散草稿模型进一步并行化了提议生成,但更大的提议容量或更高的平均接受token数(MAT)并不一定能带来更高的吞吐量。由于验证仅接受连续前缀,早期拒绝会导致后续正确预测失效,限制了MAT的提升;而如果每轮延迟增加,即使MAT更高也无法提升吞吐量。我们提出DScout,它使用两种互补的因子分解来引导扩散草稿模型的训练与推理,以获得更高的吞吐量。对于训练,DScout将位置损失分解为包含损失、权重损失和预测损失,这指导了混合视野训练、块内未掩码操作和接受加权损失的设计与组合,使监督与前缀接受对齐并提升MAT。对于推理,DScout将吞吐量分解为MAT与每轮延迟的比值,指导目标条件深度、草稿模型深度和推理块大小的选择,为每个工作负载确定高吞吐量配置。在六个基准测试中,DScout在Qwen3-4B和Qwen3-8B上分别实现了比自回归解码高3.84倍和4.14倍的平均加速,将比现有最优基线的平均加速分别提升了9.1%和7.1%。在不同模型和并发设置下,我们的推理配置策略比DScout的固定配置提供了最高9.75%的额外平均加速。

英文摘要:

Speculative decoding accelerates large language model inference through parallel verification of tokens proposed by a drafter. Diffusion drafters further parallelize proposal generation, but greater proposal capacity or higher mean accepted tokens (MAT) need not yield higher throughput. Because verification accepts only a consecutive prefix, early rejections can render later correct predictions ineffective, limiting MAT gains, while even higher MAT may not improve throughput if per-round latency increases. We introduce DScout, which uses two complementary factorizations to guide diffusion drafter training and inference for higher throughput. For training, DScout factorizes position-wise loss into inclusion, weighting, and prediction loss. This guides the design and combination of mixed-horizon training, intra-block unmasking, and acceptance-weighted loss to align supervision with prefix acceptance and improve MAT. For inference, DScout factorizes throughput as the ratio of MAT to per-round latency, guiding target-conditioning depth, drafter depth, and inference block size selection to identify a high-throughput configuration for each workload. Across six benchmarks, DScout achieves $3.84\times$ and $4.14\times$ mean speedups over autoregressive decoding on Qwen3-4B and Qwen3-8B, respectively, improving mean speedup over the state-of-the-art baseline by 9.1% and 7.1%. Across models and concurrency settings, our inference configuration policy provides up to 9.75% additional mean speedup over DScout's fixed configuration.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑