掩码扩散模型（MDMs）通过迭代选择位置进行解屏蔽并预测这些位置的标记来生成文本。然而，MDMs缺乏适当的似然评估：证据下界（ELBO）不仅是一个对数似然的松散界，而且，如我们所展示的，它是在训练分布下计算的，而不是在测试时分布下。我们通过DUEL框架解决这一问题，该框架统一了采用确定性位置选择的领先MDM采样策略。我们证明DUEL采样器在测试时分布下允许精确的似然计算——为MDMs提供了正确的似然，从而提供了正确的困惑度，这是首次实现。这种正确的困惑度是自回归困惑度的自然类比，使我们能够重新审视MDMs的关键问题。MDMs比之前认为的要好得多：在领域内数据上，MDM与自回归困惑度的差距缩小了高达32%，在零样本基准上缩小了82%。DUEL使在计算预算下对快速并行采样器进行首次原则性比较成为可能——这种分析无法用ELBO进行，也不可靠用生成困惑度进行——识别出一个强大的默认方法。最后，对位置排序的Oracle搜索揭示MDMs可以远超自回归模型——在AG新闻上实现36.47 vs. 52.11的困惑度，证明MDM性能的上限尚未达到。
英文摘要：
Masked diffusion models (MDMs) generate text by iteratively selecting positions to unmask and then predicting tokens at those positions. Yet MDMs lack proper likelihood evaluation: the evidence lower bound (ELBO) is not only a loose bound on log-likelihood, but, as we show, is also computed under the training distribution rather than the test-time distribution. We resolve this within our DUEL framework, which unifies leading MDM sampling strategies that employ $\textit{deterministic}$ position selection. We prove that DUEL samplers admit $\textbf{exact likelihood computation under the test-time distribution}$ -- giving MDMs $\textit{proper}$ likelihood, and hence proper perplexity, for the first time. This proper perplexity is the natural analogue of autoregressive perplexity and lets us revisit key questions about MDMs. $\textbf{MDMs are substantially better than previously thought}$: the MDM-autoregressive perplexity gap shrinks by up to $32\%$ on in-domain data and $82\%$ on zero-shot benchmarks. DUEL enables the first principled comparison of fast,parallel samplers across compute budgets -- an analysis impossible with the ELBO and unreliable with generative perplexity -- identifying a strong default method. Finally, oracle search over position orderings reveals MDMs can far surpass autoregressive models -- achieving $36.47$ vs. $52.11$ perplexity on AG News -- demonstrating the ceiling of MDM performance has not yet been reached.
Harvard University（哈佛大学）；Kempner Institute（凯普纳研究所）；Microsoft Research New England（微软研究院新英格兰分部）；Institute for Artificial Intelligence and Fundamental Interactions, MIT（麻省理工学院人工智能与基础相互作用研究所）
Interpolating Discrete Diffusion Models with Controllable Resampling
School of Computation, Information and Technology, Technical University of Munich（计算、信息与技术学院，慕尼黑技术大学）；Munich Data Science Institute（慕尼黑数据科学研究所）；Munich Center for Machine Learning（慕尼黑机器学习中心）
Uniform Diffusion Models Revisited: Leave-One-Out Denoiser and Absorbing State Reformulation
CMAP, Ecole polytechnique（巴黎高等学院CMAP实验室）；Institute of Foundation Models（基础模型研究所）；Inria, PSL Research University（法国国家信息与自动化研究所）；MBZUAI（马尔科姆·布罗德本特大学人工智能研究所）；EPITA, LRE（EPITA实验室）
Re-evaluating Confidence Remasking in Masked Diffusion Language Models
UvA-Bosch Delta Lab, University of Amsterdam（阿姆斯特丹大学UvA-Bosch Delta实验室）；Bosch Center for AI（博世人工智能中心）；University of Basel（巴塞尔大学）；Johns Hopkins University（约翰霍普金斯大学）
Variational Autoencoding Discrete Diffusion with Enhanced Dimensional Correlations Modeling
School of Mathematical Sciences, Peking University（北京大学数学科学学院）；Center for Statistical Science, Peking University（北京大学统计科学中心）；Academy of Mathematics and Systems Science, Chinese Academy of Sciences（中国科学院数学与系统科学研究院）；Huawei Foundation Model Dept（华为基金会模型部门）；The Chinese University of Hong Kong, Shenzhen（香港中文大学（深圳））
Inria（法国国家信息与自动化技术研究院）；PSL Research University（巴黎社会科学高等研究院）；CMAP（巴黎高等理工学院应用数学与计算科学实验室）；Ecole Polytechnique Palaiseau, France（法国巴黎高等理工学院Palaiseau分校）；Sakana AI Tokyo, Japan（日本东京Sakana AI公司）
Stability-Weighted Decoding for Diffusion Language Models
Department of XXX, University of YYY, Location, Country（XXX系，YYY大学，Location，Country）；School of ZZZ, Institute of WWW, Location, Country（ZZZ学院，WWW研究所，Location，Country）；Department of Data Science and Artificial Intelligence, The Hong Kong Polytechnic University, Hong Kong, China（数据科学与人工智能系，香港理工大学，香港，中国）
One-step lowest-variance selection in a Gaussian random-field model motivated by masked diffusion: Total correlation and a square root collision threshold
