CorrFill：在扩散模型中利用对应引导增强基于参考的图像修复的忠实性
CorrFill: Enhancing Faithfulness in Reference-based Inpainting with Correspondence Guidance in Diffusion Models
AI总结：
本文提出无需训练的 CorrFill 模块，通过修复时估计的对应约束、自注意力掩码和输入张量更新，增强扩散修复对参考图像几何关系的忠实性，并提升多种基线及 SOTA 方法性能。
摘要：
在基于参考的图像修复任务中，会提供一张额外的参考图像，用于将受损目标图像恢复到其原始状态。扩散模型（尤其是 Stable Diffusion）的进展使该任务能够采用简单的表述形式。然而，现有的基于扩散的方法通常缺乏对参考图像与受损图像之间相关性的显式约束，导致修复结果对参考图像的忠实性较低。在本工作中，我们提出 CorrFill，这是一个无需训练的模块，旨在增强对参考图像与目标图像之间几何相关性的感知。该增强通过利用修复过程中估计的对应约束来引导修复过程实现，具体包括在自注意力层中使用注意力掩码，以及根据这些约束使用目标函数更新输入张量。实验结果表明，CorrFill 通过强调对参考图像的忠实性，显著提升了多种基于扩散的基线方法的性能，其中包括最先进的方法。
英文摘要：
In the task of reference-based image inpainting, an additional reference image is provided to restore a damaged target image to its original state. The advancement of diffusion models, particularly Stable Diffusion, allows for simple formulations in this task. However, existing diffusion-based methods often lack explicit constraints on the correlation between the reference and damaged images, resulting in lower faithfulness to the reference images in the inpainting results. In this work, we propose CorrFill, a training-free module designed to enhance the awareness of geometric correlations between the reference and target images. This enhancement is achieved by guiding the inpainting process with correspondence constraints estimated during inpainting, utilizing attention masking in self-attention layers and an objective function to update the input tensor according to the constraints. Experimental results demonstrate that CorrFill significantly enhances the performance of multiple baseline diffusion-based methods, including state-of-the-art approaches, by emphasizing faithfulness to the reference images.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
