arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2412.11152cs.CV

双调度反演:用于真实图像编辑的无训练、无调优反演方法

Dual-Schedule Inversion: Training- and Tuning-Free Inversion for Real Image Editing

Shenzhen Institute of Advanced Technology, Chinese Academy of Sciences(中国科学院深圳先进技术研究院)

Jiancheng Huang, Yi Huang, Jianzhuang Liu, Donghao Zhou, Yifan Liu, Shifeng Chen

AI总结:

针对DDIM反演在真实图像编辑中重建失败的问题,提出双调度反演方法,无需训练或调优即可完美重建图像,并设计分类器自适应结合编辑方法,确保编辑结果符合文本语义且未编辑部分保持原貌。

摘要:

文本条件图像编辑是一项实用的AIGC任务,近年来展现出巨大的商业和学术价值。对于真实图像编辑,大多数基于扩散模型的方法在编辑前使用DDIM反演作为第一阶段。然而,DDIM反演常常导致重建失败,进而影响下游编辑的性能。为解决这一问题,我们首先分析了DDIM反演重建失败的原因。随后,我们提出了一种新的反演和采样方法,命名为双调度反演(Dual-Schedule Inversion)。我们还设计了一个分类器,以自适应地将双调度反演与不同的编辑方法相结合,实现用户友好的图像编辑。我们的工作能够实现卓越的重建和编辑性能,具有以下优势:1)无需微调即可完美重建真实图像,且其可逆性在数学上得到保证;2)编辑后的对象/场景符合文本提示的语义;3)对象/场景中未编辑的部分保留原始身份特征。

英文摘要:

Text-conditional image editing is a practical AIGC task that has recently emerged with great commercial and academic value. For real image editing, most diffusion model-based methods use DDIM Inversion as the first stage before editing. However, DDIM Inversion often results in reconstruction failure, leading to unsatisfactory performance for downstream editing. To address this problem, we first analyze why the reconstruction via DDIM Inversion fails. We then propose a new inversion and sampling method named Dual-Schedule Inversion. We also design a classifier to adaptively combine Dual-Schedule Inversion with different editing methods for user-friendly image editing. Our work can achieve superior reconstruction and editing performance with the following advantages: 1) It can reconstruct real images perfectly without fine-tuning, and its reversibility is guaranteed mathematically. 2) The edited object/scene conforms to the semantics of the text prompt. 3) The unedited parts of the object/scene retain the original identity.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑