CtrlDiff：通过动态块预测与可控生成增强大型扩散语言模型
CtrlDiff: Boosting Large Diffusion Language Models with Dynamic Block Prediction and Controllable Generation
AI总结：
针对大型扩散语言模型固定粒度、可控性弱的问题，提出动态可控半自回归框架CtrlDiff，结合强化学习动态分块与分类器引导控制，提升了混合扩散模型性能，缩小了与最优自回归方法的差距，支持多任务条件文本生成。
摘要：
尽管自回归模型近年来在语言建模领域占据主导地位，但学界对探索传统下一词元预测框架的替代范式的兴趣日益浓厚。基于扩散的语言模型凭借强大的并行生成能力与内在可编辑性，已成为极具吸引力的替代方案。然而，这类模型往往受限于固定长度生成。一个颇具前景的方向是结合两种范式的优势：将序列分割为块，对块间的自回归依赖关系建模，同时利用离散扩散估计给定前文上下文时每个块内的条件分布。尽管如此，其实际应用常受两大关键局限阻碍：僵化的固定长度输出，以及缺乏灵活的控制机制。\n在本研究中，我们针对当前大型扩散语言模型存在的固定粒度与可控性弱这一关键局限展开研究。我们提出CtrlDiff——一种动态可控的半自回归框架，该框架借助强化学习，基于局部语义自适应确定每个生成块的大小。此外，我们引入了一种适配离散扩散的分类器引导控制机制，在大幅降低计算开销的同时，无需重新训练即可实现高效的事后条件调控。大量实验表明，CtrlDiff树立了混合扩散模型的新标杆，缩小了与当前最优自回归方法的性能差距，并且能够在多种任务中实现有效的条件文本生成。
英文摘要：
Although autoregressive models have dominated language modeling in recent years, there has been a growing interest in exploring alternative paradigms to the conventional next-token prediction framework. Diffusion-based language models have emerged as a compelling alternative due to their powerful parallel generation capabilities and inherent editability. However, these models are often constrained by fixed-length generation. A promising direction is to combine the strengths of both paradigms, segmenting sequences into blocks, modeling autoregressive dependencies across blocks while leveraging discrete diffusion to estimate the conditional distribution within each block given the preceding context. Nevertheless, their practical application is often hindered by two key limitations: rigid fixed-length outputs and a lack of flexible control mechanisms. In this work, we address the critical limitations of fixed granularity and weak controllability in current large diffusion language models. We propose CtrlDiff, a dynamic and controllable semi-autoregressive framework that adaptively determines the size of each generation block based on local semantics using reinforcement learning. Furthermore, we introduce a classifier-guided control mechanism tailored to discrete diffusion, which significantly reduces computational overhead while facilitating efficient post-hoc conditioning without retraining. Extensive experiments demonstrate that CtrlDiff sets a new standard among hybrid diffusion models, narrows the performance gap to state-of-the-art autoregressive approaches, and enables effective conditional text generation across diverse tasks.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
