面向文本引导潜在扩散图像编辑的对比去噪分数
Contrastive Denoising Score for Text-guided Latent Diffusion Image Editing
AI总结：
本文提出对比去噪分数（CDS），通过在DDS框架中引入CUT损失并利用潜在扩散模型自注意力层特征，实现零样本图像转换与NeRF编辑中的结构对应和内容可控。
摘要：
随着文本到图像扩散模型的显著发展，图像编辑方法变得更加多样并持续演进。该领域中一种近期颇具前景的方法是Delta去噪分数（DDS）——一种基于分数蒸馏采样（SDS）框架的图像编辑技术，利用文本到图像扩散模型丰富的生成先验。然而，仅依赖评分函数之间的差异不足以保留原始图像中的特定结构元素，而这是图像编辑的一个关键方面。为解决这一问题，本文提出了一种对DDS极其简单但非常有效的改进，称为对比去噪分数（CDS），用于潜在扩散模型（LDM）。受DDS与非配对图像到图像转换对比学习（CUT）之间异同的启发，我们引入了一种在DDS框架内使用CUT损失的直接方法。我们没有像原始CUT方法那样采用辅助网络，而是利用LDM的中间特征，特别是来自自注意力层的特征，这些特征具有丰富的空间信息。我们的方法能够实现零样本图像到图像转换和神经辐射场（NeRF）编辑，在保持内容可控性的同时实现输入与输出之间的结构对应。定性结果和比较证明了所提方法的有效性。项目页面：https://hyelinnam.github.io/CDS/
英文摘要：
With the remarkable advent of text-to-image diffusion models, image editing methods have become more diverse and continue to evolve. A promising recent approach in this realm is Delta Denoising Score (DDS) - an image editing technique based on Score Distillation Sampling (SDS) framework that leverages the rich generative prior of text-to-image diffusion models. However, relying solely on the difference between scoring functions is insufficient for preserving specific structural elements from the original image, a crucial aspect of image editing. To address this, here we present an embarrassingly simple yet very powerful modification of DDS, called Contrastive Denoising Score (CDS), for latent diffusion models (LDM). Inspired by the similarities and differences between DDS and the contrastive learning for unpaired image-to-image translation(CUT), we introduce a straightforward approach using CUT loss within the DDS framework. Rather than employing auxiliary networks as in the original CUT approach, we leverage the intermediate features of LDM, specifically those from the self-attention layers, which possesses rich spatial information. Our approach enables zero-shot image-to-image translation and neural radiance field (NeRF) editing, achieving structural correspondence between the input and output while maintaining content controllability. Qualitative results and comparisons demonstrates the effectiveness of our proposed method. Project page: https://hyelinnam.github.io/CDS/
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
