arXivDaily arXiv每日学术速递 周一至周五更新
北美退伍军人节(Veterans Day),12 日无更新
arXiv 2410.02416cs.LGcs.CV
收藏

消除扩散模型中高引导尺度下的过饱和与伪影

Eliminating Oversaturation and Artifacts of High Guidance Scales in Diffusion Models

ETH Zürich(苏黎世联邦理工学院);DisneyResearch|Studios(迪士尼研究工作室)

Seyedmorteza Sadat, Otmar Hilliges, Romann M. Weber

AI总结:

本文提出自适应投影引导(APG),通过降权 CFG 更新中的平行分量并引入重新缩放与动量方法,在更高引导尺度下消除过饱和和伪影,同时保持生成质量,是一种即插即用的 CFG 替代方案。

摘要:

无分类器引导(CFG)对于提升扩散模型的生成质量以及输入条件与最终输出之间的一致性至关重要。虽然通常需要较高的引导尺度来增强这些方面,但它也会导致过饱和和不真实的伪影。在本文中,我们重新审视了 CFG 更新规则,并引入修改来解决这一问题。我们首先将 CFG 中的更新项相对于条件模型预测分解为平行分量和正交分量,并观察到平行分量主要导致过饱和,而正交分量则提升图像质量。据此,我们提出对平行分量进行降权,以在不产生过饱和的情况下实现高质量生成。此外,我们将 CFG 与梯度上升建立联系,并基于这一见解为 CFG 更新规则引入了一种新的重新缩放和动量方法。我们的方法被称为自适应投影引导(APG),保留了 CFG 的质量提升优势,同时能够在更高的引导尺度下使用而不产生过饱和。APG 易于实现,并且实际上不会给采样过程增加额外的计算开销。通过大量实验,我们证明 APG 与各种条件扩散模型和采样器兼容,在保持与 CFG 相当的精确度的同时,改善了 FID、召回率和饱和度分数,使我们的方法成为标准无分类器引导的优越即插即用替代方案。

英文摘要:

Classifier-free guidance (CFG) is crucial for improving both generation quality and alignment between the input condition and final output in diffusion models. While a high guidance scale is generally required to enhance these aspects, it also causes oversaturation and unrealistic artifacts. In this paper, we revisit the CFG update rule and introduce modifications to address this issue. We first decompose the update term in CFG into parallel and orthogonal components with respect to the conditional model prediction and observe that the parallel component primarily causes oversaturation, while the orthogonal component enhances image quality. Accordingly, we propose down-weighting the parallel component to achieve high-quality generations without oversaturation. Additionally, we draw a connection between CFG and gradient ascent and introduce a new rescaling and momentum method for the CFG update rule based on this insight. Our approach, termed adaptive projected guidance (APG), retains the quality-boosting advantages of CFG while enabling the use of higher guidance scales without oversaturation. APG is easy to implement and introduces practically no additional computational overhead to the sampling process. Through extensive experiments, we demonstrate that APG is compatible with various conditional diffusion models and samplers, leading to improved FID, recall, and saturation scores while maintaining precision comparable to CFG, making our method a superior plug-and-play alternative to standard classifier-free guidance.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑