arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2210.14784cs.LG
收藏

带单纯形扩散的范畴偏微分方程

Categorical SDEs with Simplex Diffusion

DeepMind(深度思维)

Pierre H. Richemond, Sander Dieleman, Arnaud Doucet

摘要:

扩散模型通常通过产生连续值数据点在生成建模的标准框架中操作。为此,他们依赖于一个渐进的高斯平滑的原始数据分布,这承认一个标准的布朗运动的增量涉及到一个渐进的解释。然而,诸如文本生成或强化学习之类的一些应用可能自然地通过扩散分类值数据来更好地服务,即,将扩散提升到概率分布的空间。为此,这篇简短的理论笔记提出了单纯形扩散,一种直接扩散位于n维概率单纯形上的数据点的方法。我们展示了这与单纯形上的狄利克雷分布的关系,以及由于以前在经济学和数学金融学中使用的多维Cox-Ingersoll-Ross过程(缩写为CIR),类似的Dirichlet分布是如何实现的。最后,我们的评论CIR过程的轨迹的数值实现,并讨论了我们的方法的一些局限性。

英文摘要:

Diffusion models typically operate in the standard framework of generative modelling by producing continuously-valued datapoints. To this end, they rely on a progressive Gaussian smoothing of the original data distribution, which admits an SDE interpretation involving increments of a standard Brownian motion. However, some applications such as text generation or reinforcement learning might naturally be better served by diffusing categorical-valued data, i.e., lifting the diffusion to a space of probability distributions. To this end, this short theoretical note proposes Simplex Diffusion, a means to directly diffuse datapoints located on an n-dimensional probability simplex. We show how this relates to the Dirichlet distribution on the simplex and how the analogous SDE is realized thanks to a multi-dimensional Cox-Ingersoll-Ross process (abbreviated as CIR), previously used in economics and mathematical finance. Finally, we make remarks as to the numerical implementation of trajectories of the CIR process, and discuss some limitations of our approach.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑