We develop a framework for non-asymptotic analysis of deterministic samplers used for diffusion generative modeling. Several recent works have analyzed stochastic samplers using tools like Girsanov's theorem and a chain rule variant of the interpolation argument. Unfortunately, these techniques give vacuous bounds when applied to deterministic samplers. We give a new operational interpretation for deterministic sampling by showing that one step along the probability flow ODE can be expressed as two steps: 1) a restoration step that runs gradient ascent on the conditional log-likelihood at some infinitesimally previous time, and 2) a degradation step that runs the forward process using noise pointing back towards the current iterate. This perspective allows us to extend denoising diffusion implicit models to general, non-linear forward processes. We then develop the first polynomial convergence bounds for these samplers under mild conditions on the data distribution.
Optimal Convergence Analysis of DDPM for General Distributions
Department of Statistics and Data Science, Chinese University of Hong Kong（香港中文大学统计与数据科学系）；Department of Statistics, University of Illinois Urbana-Champaign（伊利诺伊大学厄巴纳-香槟分校统计系）
Unified Convergence Analysis for Score-Based Diffusion Models with Deterministic Samplers
School of Mathematical Science, Peking University（北京大学数学科学学院）；Department of Computer Science, University of California, Los Angeles（加利福尼亚大学洛杉矶分校计算机科学系）
Diffusion models have emerged as an expressive family of generative models rivaling GANs in sample quality and autoregressive models in likelihood scores. Standard diffusion models typically require hundreds of forward passes through the model to generate a single high-fidelity sample. We introduce Differentiable Diffusion Sampler Search (DDSS): a method that optimizes fast samplers for any pre-trained diffusion model by differentiating through sample quality scores. We also present Generalized Gaussian Diffusion Models (GGDM), a family of flexible non-Markovian samplers for diffusion models. …
Massachusetts Institute of Technology（麻省理工学院）；University of Oxford（牛津大学）；MIT-IBM Watson AI Lab, IBM Research（IBM研究院麻省理工学院-IBM沃森人工智能实验室）；Toyota Research Institute（丰田研究院）；Meta Reality Labs Research（Meta现实实验室）
本文揭示了 SDS 过度平滑的原因在于其高方差噪声采样，并提出在每个更新步骤中通过反演 DDIM 恢复更好的噪声近似，从而显著提升 3D 生成质量且无需额外训练。
Asia University（亚细亚大学）；PKSHA Technology, Inc.（PKSHA科技公司）
Diffusion generative models have emerged as a new challenger to popular deep neural generative models such as GANs, but have the drawback that they often require a huge number of neural function evaluations (NFEs) during synthesis unless some sophisticated sampling strategies are employed. This paper proposes new efficient samplers based on the numerical schemes derived by the familiar Taylor expansion, which directly solves the ODE/SDE of interest. In general, it is not easy to compute the derivatives that are required in higher-order Taylor schemes, but in the case of diffusion models, this …
Accelerating Redshift-Conditioned Galaxy Image Synthesis with One-step Generative Modeling
The Center for Computational Science（计算科学中心）；University College London（伦敦大学学院）；Cavendish Laboratory（卡文迪许实验室）；Kavli Institute for Cosmology University of Cambridge（剑桥大学卡文迪许宇宙研究所）
MaRS: A Fast Sampler for Mean Reverting Diffusion based on ODE and SDE Solvers
School of Artificial Intelligence, University of Chinese Academy of Sciences（中国科学院大学人工智能学院）；Institute of Automation, Chinese Academy of Sciences (CASIA)（中国科学院自动化研究所）；DAMO Academy, Alibaba Group（阿里巴巴达摩院）；Hupan Laboratory（湖畔实验室）
Decoupled Data Consistency with Diffusion Purification for Image Restoration
Department of Electrical Engineering and Computer Science, University of Michigan（电气工程与计算机科学系，密歇根大学）；Departments of Computational Mathematics, Science and Engineering, and Biomedical Engineering, Michigan State University（计算数学、科学与工程系，以及生物医学工程系，密歇根州立大学）
