通过均匀化实现精确实现:离散扩散模型:离散扩散模型的收敛性分析
Convergence Analysis of Discrete Diffusion Model: Exact Implementation through Uniformization
AI总结:
本文研究离散扩散模型的理论性质,提出基于连续马尔可夫链均匀化的算法,在合理假设下推导出全变差距离和KL散度保证,证明了离散扩散模型的优势。
摘要:
扩散模型在数据生成任务中取得了巨大的经验成功。最近,一些努力将扩散模型框架适应离散状态空间,为建模本质上离散数据(如语言和图))提供了更自然的方法。这是通过将前向加噪过程和相应的反向过程表述为连续时间马尔可夫链(CTMCs)来实现的。本文研究了离散扩散模型的理论性质。具体而言,引入了一种利用连续马尔可夫链均匀化的算法,在随机时间点上实现转移。在离散得分函数学习的合理假设下,推导出了从超立方体上任意分布采样的全变差距离距离和KL散度保证。结果与$\mathbb{R}^d$扩散模型的最先进成果一致致致,并进一步强调了离散扩散模型相较于$\mathbb{R}^d$设置的优势。
英文摘要:
Diffusion models have achieved huge empirical success in data generation tasks. Recently, some efforts have been made to adapt the framework of diffusion models to discrete state space, providing a more natural approach for modeling intrinsically discrete data, such as language and graphs. This is achieved by formulating both the forward noising process and the corresponding reversed process as Continuous Time Markov Chains (CTMCs). In this paper, we investigate the theoretical properties of the discrete diffusion model. Specifically, we introduce an algorithm leveraging the uniformization of continuous Markov chains, implementing transitions on random time points. Under reasonable assumptions on the learning of the discrete score function, we derive Total Variation distance and KL divergence guarantees for sampling from any distribution on a hypercube. Our results align with state-of-the-art achievements for diffusion models in $\mathbb{R}^d$ and further underscore the advantages of discrete diffusion models in comparison to the $\mathbb{R}^d$ setting.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。