发表机构
McGill University; Mila - Quebec AI Institute; University of Cambridge; University of Toronto; MBZUAI - Mohamed bin Zayed University of Artificial Intelligence; Tsinghua University; Rochester Institute of Technology; Salesforce; University of Illinois Chicago（麦吉尔大学; 米拉-魁北克人工智能研究所; 剑桥大学; 多伦多大学; 穆罕默德·本·扎耶德人工智能大学; 清华大学; 罗彻斯特理工学院; Salesforce公司; 伊利诺伊大学芝加哥分校）
机构由 AI 辅助整理，请以论文原文为准。AI 中文总结
研究离散扩散模型，引入统一框架从离散状态空间构建审视该模型，让现有公式成为共同设计空间实例，揭示训练、推理等方面权衡，为未来研究提供方向。
AI 中文摘要
离散去噪扩散模型（DDMs）最近成为离散数据自回归建模的有力替代方案，具有并行生成和迭代全局细化能力。与连续扩散不同，离散扩散模型的状态空间由离散状态空间的构建方式决定。本文引入统一概念框架，通过构建底层离散状态空间来审视离散扩散模型。在此框架下，现有公式成为共同设计空间的不同实例，还揭示了训练目标、推理算法等方面的常见权衡，为未来研究指明方向。
英文摘要
Discrete denoising diffusion models (DDMs) have recently emerged as a compelling alternative to autoregressive (AR) modeling for discrete data, offering parallel generation and iterative global refinement capabilities. Unlike continuous diffusion, where the state space is fixed, DDMs are fundamentally shaped by how the discrete state space is constructed: the tokenization scheme, the vocabulary topology, and domain-specific structural alphabets. This work introduces a unified conceptual framework that views discrete diffusion models through the construction of the underlying discrete state space. Within this framework, existing formulations, including transition-matrix, masking/absorbing-state, and score/ratio-based approaches, emerge as different instantiations of a common design space. The framework further exposes common design trade-offs across training objectives, inference algorithms, scaling behavior, systems optimization, and evaluation protocols, suggesting several promising directions for future research.
