通过轨迹自蒸馏实现少步扩散语言模型
Few-Step Diffusion Language Models via Trajectory Self-Distillation
AI总结:
本文提出通过轨迹自蒸馏框架训练少步学生模型,以匹配全步教师的生成轨迹,从而减少解码步骤带来的输出质量下降,并结合DDO提升推理任务性能。
摘要:
扩散大型语言模型(DLLMs)已作为一种强大的生成模型出现,通过并行解码实现快速文本生成。然而,实际应用中减少解码步骤会因令牌因子化误差导致输出质量显著下降。为此,我们提出了一种自蒸馏框架,训练少步学生模型以匹配全步教师的生成轨迹。我们理论和实验证明,轨迹级监督可以缓解这种因子化误差,从而实现有效的少步解码。我们进一步结合直接判别优化(DDO),一种鼓励向教师模式寻求的反向KL目标,从而在具有挑战性的推理任务上获得更强的性能。在推理和代码生成基准测试中,我们的方法显著缩小了少步解码与全步解码之间的差距。源代码可在https://github.com/Tyrion58/T3D获取。
英文摘要:
Diffusion large language models (DLLMs) have emerged as powerful generative models with the promise of fast text generation through parallel decoding. However, realizing this potential in practice remains challenging: reducing the number of decoding steps, typically causes a substantial degradation in output quality due to token factorization error. To alleviate this, we propose a self-distillation framework that trains a few-step student to match the generative trajectory of a full-step teacher. We theoretically and empirically show that trajectory-level supervision mitigates this factorization error, thereby enabling effective few-step decoding. We further incorporate Direct Discriminative Optimization (DDO), a reverse-KL objective that encourages mode-seeking toward the teacher's modes, yielding stronger performance on challenging reasoning tasks. Across reasoning and code-generation benchmarks, our method substantially narrows the gap between few-step and full-step decoding. The source code is available at https://github.com/Tyrion58/T3D.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。