arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2201.06503cs.LG
收藏

Analytic-DPM:扩散概率模型中最优反向方差的解析估计

Analytic-DPM: an Analytic Estimate of the Optimal Reverse Variance in Diffusion Probabilistic Models

Tsinghua University(清华大学);Renmin University of China(中国人民大学);Huawei Noah’s Ark Lab(华为诺亚方舟实验室)

Fan Bao, Chongxuan Li, Jun Zhu, Bo Zhang

摘要:

扩散概率模型(DPM)代表了一类功能强大的生成模型。尽管他们的成功,DPM的推理是昂贵的,因为它通常需要在数千个时间步长上进行迭代。该方法的一个关键问题是估计逆过程中各时间步的方差。在这项工作中,我们提出了一个令人惊讶的结果,无论是最佳的反向方差和相应的最佳KL分歧的一个共同点有解析形式w.r.t.其评分功能。在此基础上,我们提出了Analytic-Risk,这是一个免训练的推理框架,它使用Monte Carlo方法和预训练的基于分数的模型来估计方差和KL散度的解析形式。此外,为了纠正由基于分数的模型引起的潜在偏差,我们推导出最佳方差的上下界,并剪切估计值以获得更好的结果。从经验上讲,我们的分析算法提高了各种DPM的对数似然性,产生了高质量的样本,同时获得了20倍至80倍的速度提升。

英文摘要:

Diffusion probabilistic models (DPMs) represent a class of powerful generative models. Despite their success, the inference of DPMs is expensive since it generally needs to iterate over thousands of timesteps. A key problem in the inference is to estimate the variance in each timestep of the reverse process. In this work, we present a surprising result that both the optimal reverse variance and the corresponding optimal KL divergence of a DPM have analytic forms w.r.t. its score function. Building upon it, we propose Analytic-DPM, a training-free inference framework that estimates the analytic forms of the variance and KL divergence using the Monte Carlo method and a pretrained score-based model. Further, to correct the potential bias caused by the score-based model, we derive both lower and upper bounds of the optimal variance and clip the estimate for a better result. Empirically, our analytic-DPM improves the log-likelihood of various DPMs, produces high-quality samples, and meanwhile enjoys a 20x to 80x speed up.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑