基于扩散模型的文本驱动图像编辑的提示调优反演
Prompt Tuning Inversion for Text-Driven Image Editing Using Diffusion Models
AI总结：
本文提出提示调优反演技术，通过两阶段方法实现文本驱动的图像编辑，在保持高保真度的同时实现灵活编辑，并在ImageNet上验证了优越性能。
摘要：
近年来，大规模语言-图像模型（如文本引导的扩散模型）显著提升了图像生成能力，能够在各个领域生成逼真的图像。基于这一成功，当前的图像编辑方法利用文本实现对图像的直观且多功能的修改。为了使用扩散模型编辑真实图像，必须首先将图像反演为噪声潜变量，然后根据目标文本提示从中采样编辑后的图像。然而，大多数方法缺乏以下特性之一：用户友好性（例如，需要额外的掩码或输入图像的精确描述）、对更大领域的泛化能力，或对输入图像的高保真度。在本文中，我们设计了一种准确且快速的反演技术——提示调优反演，用于文本驱动的图像编辑。具体而言，我们提出的编辑方法包括重建阶段和编辑阶段。在第一阶段，我们通过提示调优反演将输入图像的信息编码为可学习的条件嵌入。在第二阶段，我们应用无分类器引导来采样编辑后的图像，其中条件嵌入通过目标嵌入与第一阶段获得的优化嵌入之间的线性插值计算。该技术确保了我们的方法在可编辑性和对输入图像的高保真度之间的优越权衡。例如，我们可以在仅目标文本提示的引导下，改变特定对象的颜色，同时保留其原始形状和背景。在ImageNet上的大量实验表明，我们的方法相比最先进的基线具有更优越的编辑性能。
英文摘要：
Recently large-scale language-image models (e.g., text-guided diffusion models) have considerably improved the image generation capabilities to generate photorealistic images in various domains. Based on this success, current image editing methods use texts to achieve intuitive and versatile modification of images. To edit a real image using diffusion models, one must first invert the image to a noisy latent from which an edited image is sampled with a target text prompt. However, most methods lack one of the following: user-friendliness (e.g., additional masks or precise descriptions of the input image are required), generalization to larger domains, or high fidelity to the input image. In this paper, we design an accurate and quick inversion technique, Prompt Tuning Inversion, for text-driven image editing. Specifically, our proposed editing method consists of a reconstruction stage and an editing stage. In the first stage, we encode the information of the input image into a learnable conditional embedding via Prompt Tuning Inversion. In the second stage, we apply classifier-free guidance to sample the edited image, where the conditional embedding is calculated by linearly interpolating between the target embedding and the optimized one obtained in the first stage. This technique ensures a superior trade-off between editability and high fidelity to the input image of our method. For example, we can change the color of a specific object while preserving its original shape and background under the guidance of only a target text prompt. Extensive experiments on ImageNet demonstrate the superior editing performance of our method compared to the state-of-the-art baselines.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
