arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2305.19947cs.CVcs.LGstat.ML
收藏

扩散模型的几何视角

A Geometric Perspective on Diffusion Models

Zhejiang University(浙江大学);Zhejiang University of Technology(浙江工业大学)

Defang Chen, Zhenyu Zhou, Jian-Ping Mei, Chunhua Shen, Chun Chen, Can Wang

AI总结:

本文从几何视角研究了扩散模型中基于ODE的采样,揭示了准线性采样轨迹与隐式去噪轨迹的平滑连接关系,并建立了最优ODE采样与经典均值漂移算法的理论联系。

摘要:

近年来,扩散模型的有效训练和快速采样技术取得了显著进展。一个显著的进展是使用随机微分方程(SDEs)及其边缘保持的常微分方程(ODEs)在统一框架中描述数据扰动和生成建模。本文仔细研究了一种流行的方差爆炸SDE的基于ODE的采样,并揭示了其采样动力学的几个有趣结构。我们发现,数据分布和噪声分布通过一条准线性采样轨迹和另一条甚至收敛更快的隐式去噪轨迹平滑连接。同时,去噪轨迹控制着相应采样轨迹的曲率,其有限差分产生了实践中使用的各种二阶采样器。此外,我们建立了最优基于ODE的采样与经典均值漂移(寻模)算法之间的理论关系,借此可以表征扩散模型的渐近行为并识别经验分数偏差。代码可在 https://github.com/zju-pi/diff-sampler 获取。

英文摘要:

Recent years have witnessed significant progress in developing effective training and fast sampling techniques for diffusion models. A remarkable advancement is the use of stochastic differential equations (SDEs) and their marginal-preserving ordinary differential equations (ODEs) to describe data perturbation and generative modeling in a unified framework. In this paper, we carefully inspect the ODE-based sampling of a popular variance-exploding SDE and reveal several intriguing structures of its sampling dynamics. We discover that the data distribution and the noise distribution are smoothly connected with a quasi-linear sampling trajectory and another implicit denoising trajectory that even converges faster. Meanwhile, the denoising trajectory governs the curvature of the corresponding sampling trajectory and its finite differences yield various second-order samplers used in practice. Furthermore, we establish a theoretical relationship between the optimal ODE-based sampling and the classic mean-shift (mode-seeking) algorithm, with which we can characterize the asymptotic behavior of diffusion models and identify the empirical score deviation. Code is available at \url{https://github.com/zju-pi/diff-sampler}.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑