arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2502.07939stat.MLcs.LG
收藏

比特级离散扩散与马尔可夫概率模型:最小假设下具有尖锐收敛界的一种改进框架

Bit-Level Discrete Diffusion with Markov Probabilistic Models: An Improved Framework with Sharp Convergence Bounds under Minimal Assumptions

Department of XXX, University of YYY, Location, Country;School of ZZZ, Institute of WWW, Location, Country;Centre de Math\'ematiques Appliqu\'ees (CMAP), \'Ecole Polytechnique, Institut Polytechnique de Paris, 91120 Palaiseau, France;Department of Mathematics, University of Padova, Italy;Sierra lab, Inria, Paris, France

Le-Tuyet-Nhi Pham, Dario Shariatian, Antonio Ocello, Giovanni Conforti, Alain Durmus

AI总结:

提出离散马尔可夫概率模型(DMPMs),在离散比特空间进行连续时间马尔可夫链加噪,以离散得分函数控制反向跳跃过程,在最小假设下建立收敛界,并在伯努利数据和二值MNIST上验证,性能优于现有方法。

摘要:

本文介绍了离散马尔可夫概率模型(DMPMs),一种用于离散数据生成的新型离散扩散算法。该算法在离散比特空间中运行,其中加噪过程是一个连续时间马尔可夫链,均匀随机地翻转标签。时间反转过程与前向噪声过程一样,是一个跳跃过程,其强度由经典得分函数的离散类比控制。关键的是,该强度被证明是前向过程某个函数的条件期望,这强调了与基于得分的生成模型在理论上的对齐。我们在最小假设下建立了算法的收敛界,确保了鲁棒性和效率,并通过在低维伯努利分布数据集和高维二值MNIST数据上的实验进行了验证。结果表明,在生成离散结构方面,与最先进方法相比具有竞争力的性能。这项工作弥合了理论基础与实际应用之间的差距,推动了有效且具有理论基础的离散生成建模的发展。

英文摘要:

This paper introduces Discrete Markov Probabilistic Models (DMPMs), a novel discrete diffusion algorithm for discrete data generation. The algorithm operates in discrete bit space, where the noising process is a continuous-time Markov chain that flips labels uniformly at random. The time-reversal process, like the forward noise process, is a jump process with its intensity governed by a discrete analogue of the classical score function. Crucially, this intensity is proven to be the conditional expectation of a function of the forward process, underlining theoretical alignment with score-based generative models. We establish convergence bounds for the algorithm under minimal assumptions, ensuring robustness and efficiency, which we demonstrate through experiments on low-dimensional Bernoulli-distributed datasets and high-dimensional binary MNIST data. The results highlight competitive performance in generating discrete structures compared to the state-of-the-art. This work bridges theoretical foundations and practical applications, advancing the development of effective and theoretically grounded discrete generative modeling.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑