arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2510.04147cs.CL
收藏

面向扩散大语言模型的自投机解码

Self Speculative Decoding for Diffusion Large Language Models

School of AI, Shanghai Jiao Tong University(上海交通大学人工智能学院);University of Science and Technology of China(中国科学技术大学);Xidian University(西安电子科技大学);Shanghai AI Laboratory(上海人工智能实验室);Huawei Technologies Ltd.(华为技术有限公司)

Yifeng Gao, Ziang Ji, Yuxuan Wang, Biqing Qi, Hanlin Xu, Linfeng Zhang

AI总结:

针对扩散大语言模型现有并行解码与逐步解码结果有偏差、性能下降的问题,提出无需辅助模块的自投机解码方法,利用模型自身并行预测能力和分层验证树实现无损加速,在开源模型上最高达3.46倍加速。

摘要:

基于扩散的大语言模型(dLLMs)已成为自回归模型的有力替代方案,凭借双向注意力和并行生成范式具备独特优势。然而,当前并行解码方法的生成结果与逐步解码存在偏差,可能引发性能下降,限制了其实际部署。为解决这一问题,我们提出自投机解码(Self Speculative Decoding,SSD),这是一种无损推理加速方法,无需辅助模块,直接利用dLLM自身同时充当投机解码的起草器和验证器。SSD引入自起草机制:模型生成多个位置的预测结果,再通过分层验证树在单次前向传播中完成验证。与需要独立草稿模型的传统投机解码不同,SSD利用dLLM固有的多位置并行预测能力,消除了模型冗余和内存开销。这种自投机方法使模型能在单次前向传播中逐步验证并接受多个token。我们的实验表明,在LLaDA、Dream等开源模型上,SSD在保持输出与逐步解码完全一致的前提下,实现了最高3.46倍的加速比。代码将在GitHub上公开发布。

英文摘要:

Diffusion-based Large Language Models (dLLMs) have emerged as a competitive alternative to autoregressive models, offering unique advantages through bidirectional attention and parallel generation paradigms. However, the generation results of current parallel decoding methods deviate from stepwise decoding, introducing potential performance degradation, which limits their practical deployment. To address this problem, we propose \textbf{S}elf \textbf{S}peculative \textbf{D}ecoding (SSD), a lossless inference acceleration method that leverages the dLLM itself as both speculative decoding drafter and verifier without auxiliary modules. SSD introduces a self-drafting mechanism where the model generates predictions for multiple positions, then verifies them through hierarchical verification trees in a single forward pass. Unlike traditional speculative decoding that requires separate draft models, SSD eliminates model redundancy and memory overhead by exploiting the dLLM's inherent parallel prediction capability for multiple positions. This self-speculative approach allows the model to progressively verify and accept multiple tokens in a single forward pass. Our experiments demonstrate that SSD achieves up to 3.46$\times$ speedup while keeping the output identical to stepwise decoding on open source models such as LLaDA and Dream. Code will be made publicly available on GitHub.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑