arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2303.00848cs.LGcs.AIstat.ML
收藏

将扩散目标理解为带有简单数据增强的 ELBO

Understanding Diffusion Objectives as the ELBO with Simple Data Augmentation

Google DeepMind(谷歌DeepMind)

Diederik P. Kingma, Ruiqi Gao

AI总结:

本文揭示常用扩散模型目标等价于不同噪声水平上 ELBO 的加权积分,并在单调加权下等价于结合高斯噪声扰动数据增强的 ELBO,据此提出新加权并在高分辨率 ImageNet 上取得最优 FID。

摘要:

为了获得最高的感知质量,最先进的扩散模型通常使用看起来与最大似然和证据下界(Evidence Lower Bound,ELBO)目标非常不同的目标进行优化。在本工作中,我们揭示扩散模型目标实际上与 ELBO 密切相关。具体而言,我们表明所有常用的扩散模型目标都等价于不同噪声水平上 ELBO 的加权积分,其中权重取决于所使用的具体目标。在单调加权条件下,这种联系更加紧密:此时扩散目标等价于 ELBO 与简单数据增强(即高斯噪声扰动)相结合。我们证明该条件对许多最先进的扩散模型都成立。在实验中,我们探索了新的单调加权,并证明了其有效性,在高分辨率 ImageNet 基准上取得了最先进的 FID 分数。

英文摘要:

To achieve the highest perceptual quality, state-of-the-art diffusion models are optimized with objectives that typically look very different from the maximum likelihood and the Evidence Lower Bound (ELBO) objectives. In this work, we reveal that diffusion model objectives are actually closely related to the ELBO. Specifically, we show that all commonly used diffusion model objectives equate to a weighted integral of ELBOs over different noise levels, where the weighting depends on the specific objective used. Under the condition of monotonic weighting, the connection is even closer: the diffusion objective then equals the ELBO, combined with simple data augmentation, namely Gaussian noise perturbation. We show that this condition holds for a number of state-of-the-art diffusion models. In experiments, we explore new monotonic weightings and demonstrate their effectiveness, achieving state-of-the-art FID scores on the high-resolution ImageNet benchmark.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑