arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2305.04651cs.CV
收藏

基于丰富提示的扩散模型再生学习用于零样本图像翻译

ReGeneration Learning of Diffusion Models with Rich Prompts for Zero-Shot Image Translation

Guangdong University of Technology(广东工业大学);The University of Sydney(悉尼大学);Anhui University(安徽大学)

Yupei Lin, Sen Zhang, Xiaojun Yang, Xiao Wang, Yukai Shi

AI总结:

本文提出ReDiffuser,一种基于再生学习的零样本图像翻译方法,无需人工提示即可自动发现编辑方向,通过交叉注意力引导和协作更新策略保留原始形状,利用预训练扩散模型无需额外训练,在真实和合成图像编辑上优于现有方法。

摘要:

大规模文本到图像模型已经展示了惊人的能力,能够合成多样且高保真的图像。然而,这些模型常常受到若干限制的困扰。首先,它们要求用户为所需的图像修改提供精确且上下文相关的描述。其次,当前模型在编辑过程中可能对原始图像内容施加显著变化。在本文中,我们探索了图像到图像扩散模型(ReDiffuser)中的再生学习,该模型无需人工提示即可保留原始图像的内容,并且所需的编辑方向在文本嵌入空间中被自动发现。为了在图像编辑过程中确保形状的一致性保留,我们提出了基于再生学习的交叉注意力引导。这种新颖的方法允许在保留图像原始形状的同时增强目标域特征的表达。此外,我们引入了一种协作更新策略,该策略能够高效地保留图像的原始形状,从而在编辑过程中提高形状保留的质量和一致性。我们提出的方法利用现有的预训练文本-图像扩散模型,无需任何额外训练。大量实验表明,所提出的方法在真实和合成图像编辑方面均优于现有工作。

英文摘要:

Large-scale text-to-image models have demonstrated amazing ability to synthesize diverse and high-fidelity images. However, these models are often violated by several limitations. Firstly, they require the user to provide precise and contextually relevant descriptions for the desired image modifications. Secondly, current models can impose significant changes to the original image content during the editing process. In this paper, we explore ReGeneration learning in an image-to-image Diffusion model (ReDiffuser), that preserves the content of the original image without human prompting and the requisite editing direction is automatically discovered within the text embedding space. To ensure consistent preservation of the shape during image editing, we propose cross-attention guidance based on regeneration learning. This novel approach allows for enhanced expression of the target domain features while preserving the original shape of the image. In addition, we introduce a cooperative update strategy, which allows for efficient preservation of the original shape of an image, thereby improving the quality and consistency of shape preservation throughout the editing process. Our proposed method leverages an existing pre-trained text-image diffusion model without any additional training. Extensive experiments show that the proposed method outperforms existing work in both real and synthetic image editing.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑