Standard decoding in Masked Diffusion Models (MDMs) is hindered by context rigidity: tokens are retained based on transient high confidence, often ignoring that early predictions lack full context. This creates cascade effects where initial inconsistencies misguide the remaining generation. Existing revision strategies attempt to mitigate this by relying on static confidence scores, but these signals are inherently myopic; inconsistent tokens can appear confident to the model itself. We propose Context-Robust Remasking (CORE), a training-free framework for inference-time revision. Rather than trusting static token probabilities, CORE identifies context-brittle tokens by probing their sensitivity to targeted masked-context perturbations. We formalize revision as a robust optimization objective over context shifts and efficiently approximate this objective to prioritize unstable tokens for revision. On LLaDA-8B-Base, CORE delivers consistent improvements across reasoning and code benchmarks, outperforming compute-matched baselines and improving MBPP by up to 9.2 percentage points.
Re-evaluating Confidence Remasking in Masked Diffusion Language Models
UvA-Bosch Delta Lab, University of Amsterdam(阿姆斯特丹大学UvA-Bosch Delta实验室);Bosch Center for AI(博世人工智能中心);University of Basel(巴塞尔大学);Johns Hopkins University(约翰霍普金斯大学)
Uniform Diffusion Models Revisited: Leave-One-Out Denoiser and Absorbing State Reformulation
CMAP, Ecole polytechnique(巴黎高等学院CMAP实验室);Institute of Foundation Models(基础模型研究所);Inria, PSL Research University(法国国家信息与自动化研究所);MBZUAI(马尔科姆·布罗德本特大学人工智能研究所);EPITA, LRE(EPITA实验室)
A Tale of Two Temperatures: Simple, Efficient, and Diverse Sampling from Diffusion Language Models
Massachusetts Institute of Technology(麻省理工学院);UvA-Bosch Delta Lab, University of Amsterdam(阿姆斯特丹大学UvA-Bosch Delta实验室);Johns Hopkins University(约翰霍普金斯大学);University of California, Irvine(加州大学尔湾分校)
RIKEN Center for Advanced Intelligence Project (AIP)(理化学研究所先进智能中心);RIKEN Center for Interdisciplinary Theoretical and Mathematical Sciences (iTHEMS)(理化学研究所跨学科理论与数学科学中心);Keio University(庆应义塾大学);National Institute of Technology, Matsue College(松江工业高等专门学校);Institute of Systems and Information Engineering, University of Tsukuba(筑波大学系统与信息工程研究所)
Step-Wise Refusal Dynamics in Autoregressive and Diffusion Language Models
Department of Computer Science, Technion – Israel Institute of Technology(技术学院计算机科学系,以色列技术学院);INSIGHT Lab, School of Electrical and Computer Engineering, Ben-Gurion University of the Negev, Israel(内斯坦实验室,贝内-加隆大学内加尔分校,以色列);Computer Science Department, University of Haifa, Haifa, Israel(海法大学计算机科学系,海法,以色列)