arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2302.10688cs.LGcs.CVstat.ML
收藏

关于扩散概率模型的校准

On Calibrating Diffusion Probabilistic Models

Department of Computer Science, Tsinghua University(清华大学计算机科学与技术系);Sea AI Lab, Singapore(Sea AI实验室,新加坡);Qing Yuan Research Institute, Shanghai Jiao Tong University(上海交通大学清源研究院)

Tianyu Pang, Cheng Lu, Chao Du, Min Lin, Shuicheng Yan, Zhijie Deng

AI总结:

本文利用数据分数反向过程的鞅性质,提出一种只需执行一次的扩散概率模型校准方法,可降低分数匹配损失并提高模型似然下界。

摘要:

近年来,扩散概率模型(DPMs)在各类生成任务中取得了令人瞩目的成果。典型的DPM框架包含一个逐步扩散数据分布的前向过程,以及一个从时间依赖的数据分数中恢复数据分布的反向过程。在本工作中,我们观察到数据分数的随机反向过程是一个鞅,由此可以推导出数据分数的集中界和可选停止定理。随后,我们发现了一种对任意预训练DPM进行校准的简单方法,该方法能够降低分数匹配损失,并因此提高模型似然的下界。我们给出了在各种模型参数化形式下的一般校准准则。我们的校准方法只需执行一次,所得模型可重复用于采样。我们在多个数据集上进行了实验,以实证验证我们的方法。我们的代码位于 https://github.com/thudzj/Calibrated-DPMs。

英文摘要:

Recently, diffusion probabilistic models (DPMs) have achieved promising results in diverse generative tasks. A typical DPM framework includes a forward process that gradually diffuses the data distribution and a reverse process that recovers the data distribution from time-dependent data scores. In this work, we observe that the stochastic reverse process of data scores is a martingale, from which concentration bounds and the optional stopping theorem for data scores can be derived. Then, we discover a simple way for calibrating an arbitrary pretrained DPM, with which the score matching loss can be reduced and the lower bounds of model likelihood can consequently be increased. We provide general calibration guidelines under various model parametrizations. Our calibration method is performed only once and the resulting models can be used repeatedly for sampling. We conduct experiments on multiple datasets to empirically validate our proposal. Our code is at https://github.com/thudzj/Calibrated-DPMs.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑