EDIT:基于训练梯度动态的早期扩散推理终止用于dLLMs
EDIT: Early Diffusion Inference Termination for dLLMs Based on Dynamics of Training Gradients
AI总结:
EDIT通过利用训练梯度动态,在保持准确性的同时减少dLLM推理步骤,提升效率。
摘要:
基于扩散的大型语言模型(dLLMs)通过迭代去噪来优化标记生成,但答案通常在所有步骤完成之前就趋于稳定。我们提出了EDIT(早期扩散推理终止),这是一种推理时的判据,能够在检测到足够的推理稳定性相对训练时的推理时,自适应地停止去噪。EDIT监控标记激活与从监督微调(SFT)期间通过AdamW聚合LoRA更新得到的推理图之间的对齐情况。在训练过程中,优化动态生成了关于参数重要性的丰富元数据,在先前的方法中通常在模型发布后被丢弃。我们保留这些信息作为学习推理路径的紧凑表示。在推理过程中,对齐分数被转换为当前去噪步骤中已解遮蔽的标记的分布,并在连续步骤之间的KL散度低于阈值时检测收敛。在多个推理基准上,EDIT将扩散步骤减少了11.8%至68.3%,在大多数情况下保持或提高了准确性,存储开销约为0.02%(大约1.5-2 MB用于所有QKV模块在8 GB模型中的32个块)。通过利用训练梯度动态,我们的工作为减少dLLM推理时间和成本开辟了新的研究方向。
英文摘要:
Diffusion-based large language models (dLLMs) refine token generations through iterative denoising, but answers often stabilize before all steps complete. We propose EDIT (Early Diffusion Inference Termination), an inference-time criterion that adaptively stops denoising once sufficient reasoning stability relative to training-time reasoning is detected. EDIT monitors the alignment between token activations and a reasoning map derived from AdamW-aggregated LoRA updates captured during supervised fine-tuning (SFT). During training, optimization dynamics generate rich metadata about parameter importance that in prior methods is typically discarded upon model release. We preserve this information as a compact representation of learned reasoning pathways. During inference, alignment scores are converted to a distribution over the tokens already unmasked at the current denoising step, and convergence is detected when KL divergence between consecutive steps falls below a threshold on the matched unmasked (visible) tokens. Across reasoning benchmarks, EDIT reduces diffusion steps by 11.8% to 68.3% while preserving or improving accuracy in most settings, with approximately 0.02% storage overhead (about 1.5-2 MB for all QKV modules across 32 blocks in an 8 GB model). By utilizing training-gradient dynamics, our work opens a new research direction for reducing dLLM inference time and cost.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。