arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2408.00735cs.CVcs.GR
收藏

TurboEdit:使用少步扩散模型的基于文本的图像编辑

TurboEdit: Text-Based Image Editing Using Few-Step Diffusion Models

Tel-Aviv University(特拉维夫大学);NVIDIA(英伟达公司)

Gilad Deutch, Rinon Gal, Daniel Garibi, Or Patashnik, Daniel Cohen-Or

AI总结:

本文提出TurboEdit,通过偏移噪声调度和伪引导方法解决少步扩散模型在文本图像编辑中的伪影与编辑强度不足问题,实现仅需三步的高效编辑。

摘要:

扩散模型为广泛的基于文本的图像编辑框架开辟了道路。然而,这些框架通常建立在扩散反向过程的多步特性之上,将其适配到蒸馏的快速采样方法已被证明极具挑战性。本文聚焦于一类流行的基于文本的编辑框架——“编辑友好”的DDPM噪声反演方法。我们分析了其在快速采样方法中的应用,并将其失败归为两类:视觉伪影的出现和编辑强度不足。我们将伪影追溯到反演噪声与预期噪声调度之间不匹配的噪声统计,并提出一种偏移噪声调度来校正该偏移。为提高编辑强度,我们提出一种伪引导方法,能够在不引入新伪影的情况下高效增大编辑幅度。总之,我们的方法仅需三步扩散即可实现基于文本的图像编辑,同时为流行基于文本编辑方法背后的机制提供了新见解。

英文摘要:

Diffusion models have opened the path to a wide range of text-based image editing frameworks. However, these typically build on the multi-step nature of the diffusion backwards process, and adapting them to distilled, fast-sampling methods has proven surprisingly challenging. Here, we focus on a popular line of text-based editing frameworks - the ``edit-friendly'' DDPM-noise inversion approach. We analyze its application to fast sampling methods and categorize its failures into two classes: the appearance of visual artifacts, and insufficient editing strength. We trace the artifacts to mismatched noise statistics between inverted noises and the expected noise schedule, and suggest a shifted noise schedule which corrects for this offset. To increase editing strength, we propose a pseudo-guidance approach that efficiently increases the magnitude of edits without introducing new artifacts. All in all, our method enables text-based image editing with as few as three diffusion steps, while providing novel insights into the mechanisms behind popular text-based editing approaches.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑