arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2510.01047cs.CVcs.AIcs.CLcs.LG
收藏

原位 Tweedie 离散扩散模型

In-Situ Tweedie Discrete Diffusion Models

Sun Yat-sen University(中山大学);Guangdong Key Laboratory of Big Data Analysis and Processing(广东大数据分析与处理重点实验室);X-Era AI Lab(X-Era人工智能实验室);Guangdong University of Technology(广东工业大学)

Xiao Li, Jiaqi Zhang, Shuxiang Zhang, Tianshui Chen, Liang Lin, Guangrun Wang

AI总结:

本研究提出原位 Tweedie 离散扩散模型,通过在离散一热空间中直接执行扩散过程,实现对离散任务的有效建模。

摘要:

尽管扩散模型在生成连续数据如图像方面表现出色,但将其应用于离散任务时,通常依赖于间接方法,这些方法要么在连续嵌入空间中操作,要么使用标记掩码机制,这两种方法都偏离了可以由 Tweedie 公式理论上保证的真正离散数据分布。我们提出了原位 Tweedie 离散扩散(TDD)框架,该框架在离散的一热空间中直接执行由 Tweedie 公式保证的扩散,因此称为“原位”。与以往在连续嵌入中扩散或掩码标记的方法不同,TDD 直接用高斯噪声腐蚀一热向量,并通过时间步条件的交叉熵目标进行迭代去噪,而不是均方误差重建。在每一步去噪过程中,模型预测类别概率,应用 argmax 得到离散预测,将其转换为一热向量,并将它们输入下一次迭代,逐步减少噪声。这一过程自然地在单一框架下统一了判别分类和生成建模。实验表明,TDD 在图像分类和文本生成任务中均表现出色,广泛的消融研究证实了每个设计组件的有效性。我们的工作建立了一种原理上处理离散扩散的方法,该方法在离散空间中运行,同时保留了扩散模型的核心特性。

英文摘要:

While diffusion models excel at generating continuous data such as images, adapting them to discrete tasks has relied on indirect approaches that either operate in continuous embedding spaces or use token masking mechanisms, both of which deviate from modeling the true discrete data distribution that can be theoretically guaranteed by Tweedie's formula. We propose in-situ Tweedie Discrete Diffusion (TDD), a framework that performs diffusion guaranteed by Tweedie's formula directly within the discrete one-hot space, hence "in-situ." Unlike prior methods that diffuse continuous embeddings or mask tokens, TDD directly corrupts one-hot vectors with Gaussian noise and performs iterative denoising through a timestep-conditioned cross-entropy objective rather than mean-squared-error reconstruction. At each denoising step, the model predicts class probabilities, applies argmax to obtain discrete predictions, converts them to one-hot vectors, and feeds them into the next iteration with progressively reduced noise. This process naturally unifies discriminative classification and generative modeling under a single framework. Experiments demonstrate that TDD achieves strong performance on both image classification and text generation tasks, with extensive ablation studies confirming the effectiveness of each design component. Our work establishes a principled approach to discrete diffusion that preserves the core characteristics of diffusion models while operating natively in discrete space.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑