iEdit：基于弱监督的局部文本引导图像编辑
iEdit: Localised Text-guided Image Editing with Weak Supervision
AI总结：
提出iEdit，一种基于弱监督的局部文本引导图像编辑方法，利用自动构建的LAION-5B数据集和分割掩码损失，在保持源图像保真度的同时实现高质量编辑。
摘要：
扩散模型（DMs）可以利用大规模数据集，通过文本引导生成逼真的图像。然而，它们在生成图像的输出空间上表现出有限的可控性。我们提出了一种新颖的文本引导图像编辑学习方法，即 iEdit，该方法以源图像和文本编辑提示为条件生成图像。由于不存在带有目标图像的完全标注数据集，以往的方法在测试时进行特定主体的微调，或采用无目标图像的对比学习，这导致在保持源图像保真度方面存在问题。我们提出自动构建一个源自 LAION-5B 的数据集，其中包含伪目标图像及其描述性编辑提示，给定输入图像-标题对。该数据集使我们能够灵活引入弱监督损失函数，以源图像的潜在噪声为条件，基于编辑提示生成伪目标图像。为了鼓励局部编辑并保留或修改图像中的空间结构，我们提出了一种损失函数，该函数在训练期间（可选地在推理时）使用分割掩码来引导编辑。我们的模型在构建的包含20万样本的数据集上训练，并受限于GPU资源。在图像保真度、CLIP对齐分数以及定性地编辑生成图像和真实图像方面，它显示出优于同类方法的结果。
英文摘要：
Diffusion models (DMs) can generate realistic images with text guidance using large-scale datasets. However, they demonstrate limited controllability in the output space of the generated images. We propose a novel learning method for text-guided image editing, namely \texttt{iEdit}, that generates images conditioned on a source image and a textual edit prompt. As a fully-annotated dataset with target images does not exist, previous approaches perform subject-specific fine-tuning at test time or adopt contrastive learning without a target image, leading to issues on preserving the fidelity of the source image. We propose to automatically construct a dataset derived from LAION-5B, containing pseudo-target images with their descriptive edit prompts given input image-caption pairs. This dataset gives us the flexibility of introducing a weakly-supervised loss function to generate the pseudo-target image from the latent noise of the source image conditioned on the edit prompt. To encourage localised editing and preserve or modify spatial structures in the image, we propose a loss function that uses segmentation masks to guide the editing during training and optionally at inference. Our model is trained on the constructed dataset with 200K samples and constrained GPU resources. It shows favourable results against its counterparts in terms of image fidelity, CLIP alignment score and qualitatively for editing both generated and real images.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
