arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2412.04301cs.CV

SwiftEdit:基于一步扩散的闪电般快速的文本引导图像编辑

SwiftEdit: Lightning Fast Text-Guided Image Editing via One-Step Diffusion

Qualcomm AI Research(高通人工智能研究)

Trong-Tung Nguyen, Quang Nguyen, Khoi Nguyen, Anh Tran, Cuong Pham

AI总结:

本文提出SwiftEdit,一种基于一步扩散的文本引导图像编辑工具,通过一步反演框架和带注意力重缩放机制的掩码引导编辑技术,在0.23秒内实现比多步方法快至少50倍的即时图像编辑,同时保持有竞争力的性能。

摘要:

文本引导图像编辑的最新进展利用多步基于扩散的文本到图像模型的丰富先验,使用户能够通过简单的文本输入进行图像编辑。然而,由于涉及成本高昂的多步反演和采样过程,这些方法通常无法满足现实世界和设备端应用所需的速度要求。为此,我们引入了SwiftEdit,一种简单但高效的编辑工具,可实现即时文本引导图像编辑(在0.23秒内)。SwiftEdit的进步在于其两项新颖贡献:一个一步反演框架,通过反演实现一步图像重建;以及一种带有我们提出的注意力重缩放机制的掩码引导编辑技术,用于执行局部图像编辑。我们提供了大量实验来证明SwiftEdit的有效性和效率。特别是,SwiftEdit实现了即时文本引导图像编辑,比以前的多步方法快得多(至少快50倍),同时在编辑结果中保持有竞争力的性能。我们的项目页面位于:https://swift-edit.github.io/

英文摘要:

Recent advances in text-guided image editing enable users to perform image edits through simple text inputs, leveraging the extensive priors of multi-step diffusion-based text-to-image models. However, these methods often fall short of the speed demands required for real-world and on-device applications due to the costly multi-step inversion and sampling process involved. In response to this, we introduce SwiftEdit, a simple yet highly efficient editing tool that achieve instant text-guided image editing (in 0.23s). The advancement of SwiftEdit lies in its two novel contributions: a one-step inversion framework that enables one-step image reconstruction via inversion and a mask-guided editing technique with our proposed attention rescaling mechanism to perform localized image editing. Extensive experiments are provided to demonstrate the effectiveness and efficiency of SwiftEdit. In particular, SwiftEdit enables instant text-guided image editing, which is extremely faster than previous multi-step methods (at least 50 times faster) while maintain a competitive performance in editing results. Our project page is at: https://swift-edit.github.io/

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑