arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2408.08332cs.CVcs.LG
收藏

TurboEdit:基于文本的即时图像编辑

TurboEdit: Instant text-based image editing

Adobe Research(奥多比研究院)

Zongze Wu, Nicholas Kolkin, Jonathan Brandt, Richard Zhang, Eli Shechtman

AI总结:

本研究针对少步数扩散模型的精确图像反演与解耦编辑难题,提出基于编码器的迭代反演技术TurboEdit,可通过修改文本提示属性实现快速编辑,反演仅需8次NFE、每次编辑仅4次NFE,性能优于现有多步扩散编辑方法。

摘要:

我们针对少步数扩散模型场景下的精确图像反演与解耦式图像编辑难题展开研究,提出了一种基于编码器的迭代反演技术。该反演网络以输入图像和上一步的重建图像为条件,能够引导下一步的重建结果向输入图像校正。我们证明,在少步数扩散模型中,通过以(自动生成的)详细文本提示为条件,可轻松实现解耦控制。若要对反演后的图像进行编辑,我们冻结噪声图并修改文本提示中的某一个属性(可手动修改,或通过由大语言模型LLM驱动的指令式编辑完成),即可生成与输入图像相似、仅单个属性发生改变的新图像,该方法还可控制编辑强度,并支持指令式文本提示。我们的方法可实现实时、逼真的文本引导图像编辑,仅需8次函数评估(NFE)即可完成反演(一次性成本),每次编辑仅需4次NFE。该方法不仅速度快,性能还显著优于当前最优的多步扩散编辑技术。

英文摘要:

We address the challenges of precise image inversion and disentangled image editing in the context of few-step diffusion models. We introduce an encoder based iterative inversion technique. The inversion network is conditioned on the input image and the reconstructed image from the previous step, allowing for correction of the next reconstruction towards the input image. We demonstrate that disentangled controls can be easily achieved in the few-step diffusion model by conditioning on an (automatically generated) detailed text prompt. To manipulate the inverted image, we freeze the noise maps and modify one attribute in the text prompt (either manually or via instruction based editing driven by an LLM), resulting in the generation of a new image similar to the input image with only one attribute changed. It can further control the editing strength and accept instructive text prompt. Our approach facilitates realistic text-guided image edits in real-time, requiring only 8 number of functional evaluations (NFEs) in inversion (one-time cost) and 4 NFEs per edit. Our method is not only fast, but also significantly outperforms state-of-the-art multi-step diffusion editing techniques.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑