LDEdit：通过潜在扩散模型实现通用文本引导图像编辑
LDEdit: Towards Generalized Text Guided Image Manipulation via Latent Diffusion Models
AI总结：
本文提出LDEdit，利用潜在扩散模型在低维潜在空间进行确定性前向扩散，并通过目标文本条件引导反向扩散，实现无需优化的零样本通用文本引导图像<｜begin▁of▁sentence｜>#编辑，支持语义编辑和风格迁移。
摘要：
近年来，视觉-语言模型研究发展迅速，使得基于自然语言的图像生成与编辑接口成为可能。许多现有的文本引导编辑技术仅限于特定类别的图像，并且通常需要微调才能迁移到不同的风格或领域。然而，使用单一模型并支持灵活文本输入的通用图像编辑具有很高的应用价值。近期工作通过使用预训练的视觉-语言编码器来引导在通用图像数据集上训练的生成模型，以解决这一任务。尽管该方法具有前景，但需要对每个输入进行昂贵的优化。本文提出了一种无需优化的方法，用于根据文本提示进行通用图像编辑。我们的方法利用最近的潜在扩散模型（Latent Diffusion Models，LDM）进行文本到图像生成，以实现零样本的文本引导编辑。我们在低维潜在空间中采用确定性前向扩散，只需提供目标文本作为条件来引导反向扩散过程，即可实现所需的编辑。我们将该方法称为LDEdit。我们展示了该方法在语义图像编辑和艺术风格迁移方面的适用性。我们的方法能够在多种领域上完成图像编辑，并能以直接的方式编辑多个属性。大量实验表明，与竞争基线方法相比，我们的方法具有明显优势。
英文摘要：
Research in vision-language models has seen rapid developments off-late, enabling natural language-based interfaces for image generation and manipulation. Many existing text guided manipulation techniques are restricted to specific classes of images, and often require fine-tuning to transfer to a different style or domain. Nevertheless, generic image manipulation using a single model with flexible text inputs is highly desirable. Recent work addresses this task by guiding generative models trained on the generic image datasets using pretrained vision-language encoders. While promising, this approach requires expensive optimization for each input. In this work, we propose an optimization-free method for the task of generic image manipulation from text prompts. Our approach exploits recent Latent Diffusion Models (LDM) for text to image generation to achieve zero-shot text guided manipulation. We employ a deterministic forward diffusion in a lower dimensional latent space, and the desired manipulation is achieved by simply providing the target text to condition the reverse diffusion process. We refer to our approach as LDEdit. We demonstrate the applicability of our method on semantic image manipulation and artistic style transfer. Our method can accomplish image manipulation on diverse domains and enables editing multiple attributes in a straightforward fashion. Extensive experiments demonstrate the benefit of our approach over competing baselines.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
