一个物品值得一个提示词：基于解耦控制的多功能图像编辑
An Item is Worth a Prompt: Versatile Image Editing with Disentangled Control
AI总结：
提出D-Edit框架，通过解耦图像-提示交互为物品-提示关联，支持基于图像、文本、掩码编辑及物品移除的统一多功能图像编辑。
摘要：
基于文本到图像扩散模型（DPMs）的成功，图像编辑是支持人类与AI生成内容交互的重要应用。在各种编辑方法中，由于在控制语义方面的能力和简单性，在提示空间内的编辑受到了更多关注。然而，由于扩散模型通常在描述性文本字幕上进行预训练，直接编辑文本提示中的词通常会导致生成完全不同的图像，违背了图像编辑的要求。另一方面，现有的编辑方法通常考虑引入空间掩码来保留未编辑区域的身份，但这通常被DPMs忽略，从而导致不和谐的编辑结果。针对这两个挑战，在本工作中，我们提出将全面的图像-提示交互解耦为几个物品-提示交互，每个物品链接到一个特殊学习的提示。由此产生的框架名为D-Edit，基于预训练扩散模型，解耦了交叉注意力层，并采用两步优化来建立物品-提示关联。然后可以通过操纵相应的提示，将多功能图像编辑应用于特定物品。我们在四种类型的编辑操作中展示了最先进的结果，包括基于图像、基于文本、基于掩码的编辑和物品移除，涵盖了大多数类型的编辑应用，所有这些都在一个统一的框架内。值得注意的是，D-Edit是第一个能够（1）通过掩码编辑实现物品编辑和（2）结合图像和基于文本编辑的框架。我们通过定性和定量评估，为多样化的图像集合展示了编辑结果的质量和多功能性。
英文摘要：
Building on the success of text-to-image diffusion models (DPMs), image editing is an important application to enable human interaction with AI-generated content. Among various editing methods, editing within the prompt space gains more attention due to its capacity and simplicity of controlling semantics. However, since diffusion models are commonly pretrained on descriptive text captions, direct editing of words in text prompts usually leads to completely different generated images, violating the requirements for image editing. On the other hand, existing editing methods usually consider introducing spatial masks to preserve the identity of unedited regions, which are usually ignored by DPMs and therefore lead to inharmonic editing results. Targeting these two challenges, in this work, we propose to disentangle the comprehensive image-prompt interaction into several item-prompt interactions, with each item linked to a special learned prompt. The resulting framework, named D-Edit, is based on pretrained diffusion models with cross-attention layers disentangled and adopts a two-step optimization to build item-prompt associations. Versatile image editing can then be applied to specific items by manipulating the corresponding prompts. We demonstrate state-of-the-art results in four types of editing operations including image-based, text-based, mask-based editing, and item removal, covering most types of editing applications, all within a single unified framework. Notably, D-Edit is the first framework that can (1) achieve item editing through mask editing and (2) combine image and text-based editing. We demonstrate the quality and versatility of the editing results for a diverse collection of images through both qualitative and quantitative evaluations.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
