潜在阴影：掩码扩散中的高斯-离散对偶性
Latent Shadows: The Gaussian-Discrete Duality in Masked Diffusion
AI总结：
本文提出掩码扩散对偶性及MCD框架，通过高斯过程投影和确定性轨迹构建，实现16倍推理加速且不降低生成质量。
摘要：
掩码离散扩散是一种用于高质量语言建模的主要范式，其中令牌被迭代地腐蚀到掩码状态，但其推理效率受限于缺乏确定性采样工具。尽管扩散对偶性能够为均匀模型提供确定性蒸馏，但这些方法通常在掩码模型上表现不佳，并且依赖于复杂的积分算子。相反，在掩码域中，先前的方法通常假设不存在确定性轨迹，迫使依赖于随机蒸馏。为了弥合这一差距，我们建立了显式的掩码扩散对偶性，证明掩码过程作为连续高斯过程通过一种新的最大值指数保持机制的投影而出现。此外，我们引入了掩码一致性蒸馏（MCD），一种利用这种对偶性的原理框架，以分析性地构建一致性蒸馏所需的确定性耦合轨迹，从而绕过数值ODE求解器。这一结果严格优于先前的随机蒸馏方法，实现了16倍的推理加速，而不会影响生成质量。我们的发现不仅为掩码和连续扩散之间的联系提供了坚实的理论基础，还解锁了一致性蒸馏在高性能离散生成中的全部潜力。我们的代码可在https://anonymous.4open.science/r/MCD-70FD上获得。
英文摘要：
Masked discrete diffusion is a dominant paradigm for high-quality language modeling where tokens are iteratively corrupted to a mask state, yet its inference efficiency is bottlenecked by the lack of deterministic sampling tools. While diffusion duality enables deterministic distillation for uniform models, these approaches generally underperform masked models and rely on complex integral operators. Conversely, in the masked domain, prior methods typically assume the absence of deterministic trajectories, forcing a reliance on stochastic distillation. To bridge this gap, we establish explicit Masked Diffusion Duality, proving that the masked process arises as the projection of a continuous Gaussian process via a novel maximum-value index preservation mechanism. Furthermore, we introduce Masked Consistency Distillation (MCD), a principled framework that leverages this duality to analytically construct the deterministic coupled trajectories required for consistency distillation, bypassing numerical ODE solvers. This result strictly improves upon prior stochastic distillation methods, achieving a 16$\times$ inference speedup without compromising generation quality. Our findings not only provide a solid theoretical foundation connecting masked and continuous diffusion, but also unlock the full potential of consistency distillation for high-performance discrete generation. Our code is available at https://anonymous.4open.science/r/MCD-70FD.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
