S$^2$Edit: 基于文本引导的图像编辑与精确语义和空间控制
S$^2$Edit: Text-Guided Image Editing with Precise Semantic and Spatial Control
AI总结：
S$^2$Edit通过精确语义和空间控制实现文本引导的图像编辑，解决了身份信息丢失和无关区域修改的问题。
摘要：
最近的扩散模型进展使能够通过文本引导生成和操纵高质量图像，以及从图像中学习概念成为可能。然而，直接应用现有方法到需要精细控制的编辑任务（例如人脸编辑）时，往往会导致身份信息和高频细节在编辑过程中丢失，或由于纠缠的概念而改变无关图像区域。在本文中，我们提出S$^2$Edit，一种基于预训练的文本到图像扩散模型的新方法，能够实现具有精确语义和空间控制的个性化编辑。我们首先对模型进行微调，将身份信息嵌入到可学习的文本标记中。在微调过程中，我们通过在文本特征空间中施加正交约束，将学习到的身份标记与要编辑的属性分离。为了确保身份标记只影响感兴趣的区域，我们应用对象掩膜来引导交叉注意力图。在推理过程中，我们的方法在局部化编辑的同时，能够忠实保留原始身份，通过学习到的语义解耦和空间聚焦的身份标记。广泛的实验表明，S$^2$Edit在定量和定性方面都优于最先进的方法。此外，我们还展示了S$^2$Edit在组合图像编辑应用中的几个示例，如化妆品转移。
英文摘要：
Recent advances in diffusion models have enabled high-quality generation and manipulation of images guided by texts, as well as concept learning from images. However, naive applications of existing methods to editing tasks that require fine-grained control, e.g., face editing, often lead to suboptimal solutions with identity information and high-frequency details lost during the editing process, or irrelevant image regions altered due to entangled concepts. In this work, we propose S$^2$Edit, a novel method based on a pre-trained text-to-image diffusion model that enables personalized editing with precise semantic and spatial control. We first fine-tune our model to embed the identity information into a learnable text token. During fine-tuning, we disentangle the learned identity token from attributes to be edited by enforcing an orthogonality constraint in the textual feature space. To ensure that the identity token only affects regions of interest, we apply object masks to guide the cross-attention maps. At inference time, our method performs localized editing while faithfully preserving the original identity with semantically disentangled and spatially focused identity token learned. Extensive experiments demonstrate the superiority of S$^2$Edit over state-of-the-art methods both quantitatively and qualitatively. Additionally, we showcase several compositional image editing applications of S$^2$Edit such as makeup transfer.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
