arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2401.13115cs.LG
收藏

收缩扩散概率模型

Contractive Diffusion Probabilistic Models

Columbia University(哥伦比亚大学)

Wenpin Tang, Hanyang Zhao

AI总结:

针对扩散概率模型中分数匹配可能无法保证的问题,提出收缩扩散概率模型(CDPMs),利用后向采样收缩性质缩小分数匹配与离散化误差,通过简单变换利用预训练权重且无需重训练,稳步提升了基线模型性能。

摘要:

扩散概率模型(DPMs)已成为生成建模中极具前景的技术。DPMs的成功依赖于两个要素:扩散过程的时间反转和分数匹配。鉴于分数匹配可能无法得到保证,我们提出一个新准则——DPMs设计中的后向采样收缩性质,由此产生了一类新的收缩扩散概率模型(CDPMs)。我们的核心洞察是,收缩性质可证明地缩小分数匹配误差和离散化误差,因此我们提出的CDPMs对这两种误差源均具有鲁棒性。在实际应用中,我们展示了CDPM可以通过简单的变换利用预训练DPMs的权重,且无需重新训练。我们通过在合成1维示例、Swiss Roll、MNIST、CIFAR-10 32x32和AFHQ 64x64数据集上的实验验证了我们的方法。值得注意的是,CDPM稳步提升了基线基于分数的扩散模型的性能。

英文摘要:

Diffusion probabilistic models (DPMs) have emerged as a promising technique in generative modeling. The success of DPMs relies on two ingredients: time reversal of diffusion processes and score matching. In view of possibly unguaranteed score matching, we propose a new criterion -- the contraction property of backward sampling in the design of DPMs, leading to a novel class of contractive DPMs (CDPMs). Our key insight is that, the contraction property can provably narrow score matching errors and discretization errors, thus our proposed CDPMs are robust to both sources of error. For practical use, we show that CDPM can leverage weights of pretrained DPMs by a simple transformation, and does not need retraining. We corroborated our approach by experiments on synthetic 1-dim examples, Swiss Roll, MNIST, CIFAR-10 32$\times$32 and AFHQ 64$\times$64 dataset. Notably, CDPM steadily improves the performance of baseline score-based diffusion models.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑