Diffusion Schrödinger Bridge及其在基于分数的生成模型中的应用
Diffusion Schrödinger Bridge with Applications to Score-Based Generative Modeling
AI总结:
提出Diffusion SB(DSB)方法,通过近似迭代比例拟合过程求解Schrödinger Bridge问题,在有限时间内生成样本,克服了传统前向SDE需长时间运行以逼近高斯分布的局限,并作为一种连续状态空间下的计算最优传输工具。
摘要:
逐步应用高斯噪声将复杂数据分布转换为近似高斯分布。反转这一动态过程即定义了一个生成模型。当前向加噪过程由随机微分方程(SDE)给定时,Song et al. (2021) 证明了如何使用分数匹配来估计相关逆向时间SDE的时间非齐次漂移。该方法的局限在于,前向SDE必须运行足够长的时间,才能使最终分布近似为高斯分布。相比之下,求解Schrödinger Bridge问题(SB),即路径空间上的熵正则化最优传输问题,能够产生在有限时间内从数据分布生成样本的扩散过程。我们提出了Diffusion SB(DSB),这是一种用于求解SB问题的迭代比例拟合(IPF)过程的原创近似方法,并提供了理论分析及生成建模实验。第一次DSB迭代恢复了Song et al. (2021) 提出的方法,并具有使用更短时间间隔的灵活性,因为后续的DSB迭代减少了前向(分别地,后向)SDE的最终时间边缘分布相对于先验(分别地,数据)分布的偏差。除生成建模外,DSB作为一种广泛适用的计算最优传输工具,提供了与流行的Sinkhorn算法(Cuturi, 2013)在连续状态空间下类似的类比。
英文摘要:
Progressively applying Gaussian noise transforms complex data distributions to approximately Gaussian. Reversing this dynamic defines a generative model. When the forward noising process is given by a Stochastic Differential Equation (SDE), Song et al. (2021) demonstrate how the time inhomogeneous drift of the associated reverse-time SDE may be estimated using score-matching. A limitation of this approach is that the forward-time SDE must be run for a sufficiently long time for the final distribution to be approximately Gaussian. In contrast, solving the Schrödinger Bridge problem (SB), i.e. an entropy-regularized optimal transport problem on path spaces, yields diffusions which generate samples from the data distribution in finite time. We present Diffusion SB (DSB), an original approximation of the Iterative Proportional Fitting (IPF) procedure to solve the SB problem, and provide theoretical analysis along with generative modeling experiments. The first DSB iteration recovers the methodology proposed by Song et al. (2021), with the flexibility of using shorter time intervals, as subsequent DSB iterations reduce the discrepancy between the final-time marginal of the forward (resp. backward) SDE with respect to the prior (resp. data) distribution. Beyond generative modeling, DSB offers a widely applicable computational optimal transport tool as the continuous state-space analogue of the popular Sinkhorn algorithm (Cuturi, 2013).
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。