Forgedit：通过学习和遗忘进行文本引导图像编辑
Forgedit: Text Guided Image Editing via Learning and Forgetting
AI总结：
本文提出Forgedit方法，通过视觉-语言联合优化、向量投影和遗忘机制，在30秒内重建图像并解决过拟合，在TEdBench基准上超越SOTA，实现高效文本引导图像编辑。
摘要：
文本引导的图像编辑，针对真实或合成图像，仅以原始图像本身和目标文本提示作为输入，是一项非常普遍且具有挑战性的任务。它要求编辑模型自行估计图像中哪些部分应被编辑，然后在保留原始图像特征的同时执行刚性或非刚性编辑。在本文中，我们设计了一种新颖的文本引导图像编辑方法，名为Forgedit。首先，我们提出了一种视觉-语言联合优化框架，能够在30秒内重建原始图像，比之前的SOTA方法快得多，且过拟合程度大大降低。然后，我们在扩散模型的文本嵌入空间中提出了一种新颖的向量投影机制，能够分别控制身份相似度和编辑强度。最后，我们发现了扩散模型中UNet的一个普遍性质，即UNet编码器学习空间和结构，UNet解码器学习外观和身份。利用这一性质，我们设计了遗忘机制，成功解决了在单张图像上微调扩散模型时致命且不可避免的过拟合问题，从而显著提升了扩散模型的编辑能力。我们的方法Forgedit，基于Stable Diffusion构建，在具有挑战性的文本引导图像编辑基准TEdBench上取得了新的最先进结果，在CLIP分数和LPIPS分数方面均超越了之前的SOTA方法，如基于Imagen的Imagic。代码可在https://github.com/witcherofresearch/Forgedit获取。
英文摘要：
Text-guided image editing on real or synthetic images, given only the original image itself and the target text prompt as inputs, is a very general and challenging task. It requires an editing model to estimate by itself which part of the image should be edited, and then perform either rigid or non-rigid editing while preserving the characteristics of original image. In this paper, we design a novel text-guided image editing method, named as Forgedit. First, we propose a vision-language joint optimization framework capable of reconstructing the original image in 30 seconds, much faster than previous SOTA and much less overfitting. Then we propose a novel vector projection mechanism in text embedding space of Diffusion Models, which is capable to control the identity similarity and editing strength seperately. Finally, we discovered a general property of UNet in Diffusion Models, i.e., Unet encoder learns space and structure, Unet decoder learns appearance and identity. With such a property, we design forgetting mechanisms to successfully tackle the fatal and inevitable overfitting issues when fine-tuning Diffusion Models on one image, thus significantly boosting the editing capability of Diffusion Models. Our method, Forgedit, built on Stable Diffusion, achieves new state-of-the-art results on the challenging text-guided image editing benchmark: TEdBench, surpassing the previous SOTA methods such as Imagic with Imagen, in terms of both CLIP score and LPIPS score. Codes are available at https://github.com/witcherofresearch/Forgedit
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
