离散扩散模型的非渐近收敛性：掩码与随机游动动力学
Non-Asymptotic Convergence of Discrete Diffusion Models: Masked and Random Walk dynamics
AI总结：
本文研究了三种离散扩散模型的非渐近收敛性，通过建立基于随机游动和掩码过程的收敛保证，解决了离散状态空间下扩散模型的挑战，且在无需限制估计得分的情况下提供了最优的收敛界。
摘要：
基于高斯噪声过程的连续状态空间扩散模型在实践和理论层面已被较好理解。相比之下，离散状态空间的扩散模型研究仍较少，尤其因其组合结构和近期在生成建模中的引入而带来挑战。本文为三种流行的离散扩散模型（DDMs）建立了新的、精确的收敛保证。其中两种模型针对有限状态空间，分别基于随机游动和掩码过程。第三种DDM定义在可数无限空间$\mathbb{N}^d$上，其正向过程使用漂移随机游动。对于每种模型，反向过程可通过离散得分函数描述，原则上可以估计。然而，即使拥有完美的得分访问，模拟精确的反向过程也是不可行的，必须依赖时间离散化。本文研究了欧拉型近似方法，并在数据分布的最小假设下，建立了由此产生的模型在KL散度和总变差距离上的收敛界。据我们所知，这项研究为这些噪声过程提供了最优的非渐近收敛保证，且不依赖于估计得分的有界性假设。特别是，每种方法的计算复杂度仅与维度线性相关，最多包含对数因子。
英文摘要：
Diffusion models for continuous state spaces based on Gaussian noising processes are now relatively well understood from both practical and theoretical perspectives. In contrast, results for diffusion models on discrete state spaces remain far less explored and pose significant challenges, particularly due to their combinatorial structure and their more recent introduction in generative modelling. In this work, we establish new and sharp convergence guarantees for three popular discrete diffusion models (DDMs). Two of these models are designed for finite state spaces and are based respectively on the random walk and the masking process. The third DDM we consider is defined on the countably infinite space $\mathbb{N}^d$ and uses a drifted random walk as its forward process. For each of these models, the backward process can be characterized by a discrete score function that can, in principle, be estimated. However, even with perfect access to these scores, simulating the exact backward process is infeasible, and one must rely on time discretization. In this work, we study Euler-type approximations and establish convergence bounds in both Kullback-Leibler divergence and total variation distance for the resulting models, under minimal assumptions on the data distribution. To the best of our knowledge, this study provides the optimal non-asymptotic convergence guarantees for these noising processes that do not rely on boundedness assumptions on the estimated score. In particular, the computational complexity of each method scales only linearly in the dimension, up to logarithmic factors.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
