arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2406.02507cs.CVcs.AIcs.LGcs.NEstat.ML
收藏

用扩散模型自身的较差版本引导该模型

Guiding a Diffusion Model with a Bad Version of Itself

NVIDIA(英伟达);Aalto University(阿尔托大学)

Tero Karras, Miika Aittala, Tuomas Kynkäänniemi, Jaakko Lehtinen, Timo Aila, Samuli Laine

AI总结:

提出用模型自身更小、训练较少的版本来引导扩散模型生成,在保持结果多样性的同时解耦控制图像质量,在 ImageNet 基准上创下 FID 新纪录。

摘要:

图像生成扩散模型的主要关注轴是图像质量、结果的多样性程度,以及结果与给定条件(如类标签或文本提示)的对齐程度。流行的无分类器引导方法使用无条件模型来引导条件模型,从而同时实现更好的提示对齐和更高质量的图像,代价是多样性降低。这些影响似乎本质上是交织在一起的,因此难以控制。我们得出了一个令人惊讶的观察结果:通过使用模型自身更小、训练较少的版本而不是无条件模型来引导生成,可以在不牺牲多样性程度的情况下,实现对图像质量的解耦控制。这在 ImageNet 生成方面带来了显著改进,使用公开可用的网络,在 64x64 分辨率下创下了 1.01 的 FID 记录,在 512x512 分辨率下创下了 1.25 的 FID 记录。此外,该方法同样适用于无条件扩散模型,能大幅提升其质量。

英文摘要:

The primary axes of interest in image-generating diffusion models are image quality, the amount of variation in the results, and how well the results align with a given condition, e.g., a class label or a text prompt. The popular classifier-free guidance approach uses an unconditional model to guide a conditional model, leading to simultaneously better prompt alignment and higher-quality images at the cost of reduced variation. These effects seem inherently entangled, and thus hard to control. We make the surprising observation that it is possible to obtain disentangled control over image quality without compromising the amount of variation by guiding generation using a smaller, less-trained version of the model itself rather than an unconditional model. This leads to significant improvements in ImageNet generation, setting record FIDs of 1.01 for 64x64 and 1.25 for 512x512, using publicly available networks. Furthermore, the method is also applicable to unconditional diffusion models, drastically improving their quality.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑