arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2602.11133cs.LGcs.CL
收藏

准时完成:扩散语言模型的标记级早期停止

Just on Time: Token-Level Early Stopping for Diffusion Language Models

Zakhar Kohut, Severyn Shykula, Mykola Vysotskyi, Serhii Dmytryshyn, Dmytro Khamula, Michal Zakrzewski, Damian Rynczak, Jacek Małecki, Taras Rumezhak, Volodymyr Karpiv

AI总结:

本文提出了一种无需训练的标记级早期停止方法,通过动态确定标记收敛点,减少扩散步骤并提升生成效率。

摘要:

扩散语言模型通过迭代细化生成文本,这一过程通常计算效率低下,因为许多标记在最终去噪步骤之前就已达到稳定状态。我们引入了一种无需训练的标记级早期停止方法,能够在每个位置独立地识别收敛点。我们的方法利用来自模型预测和局部上下文的轻量信号,动态确定何时可以最终化单个标记。这实现了无需任务特定微调的自适应标记冻结,显著减少了所需的扩散步骤总数。在涵盖数学推理、一般问答和科学理解等多样基准上,我们的方法在保持生成质量的同时实现了最先进的效率提升。

英文摘要:

Diffusion language models generate text through iterative refinement, a process that is often computationally inefficient because many tokens reach stability long before the final denoising step. We introduce a training-free, token-level early stopping approach that identifies convergence independently at each position. Our method leverages lightweight signals derived from the model's predictions and local context to dynamically determine when individual tokens can be finalized. This yields adaptive per-token freezing without task-specific fine-tuning, substantially reducing the total number of diffusion steps required. Across diverse benchmarks, spanning mathematical reasoning, general question answering, and scientific understanding, our approach achieves substantial efficiency gains while preserving generation quality.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑