无调优反演增强控制用于一致性图像编辑
Tuning-Free Inversion-Enhanced Control for Consistent Image Editing
AI总结：
本文提出无调优反演增强控制（TIC）方法，通过关联反演与采样特征缓解DDIM重建不一致，实现真实图像的非刚性一致性编辑，无需微调，效果优于现有方法。
摘要：
真实图像的一致性编辑是一项具有挑战性的任务，因为它需要对输入图像中的主要对象执行非刚性编辑（例如改变姿势），同时不改变其身份或属性。为了保证属性一致性，一些现有方法微调整个模型或文本嵌入以实现结构一致性，但这种方法耗时且无法执行非刚性编辑。其他工作无需调优，但其性能受到去噪扩散隐式模型（DDIM）重建质量的削弱，而DDIM重建在真实场景中经常失败。在本文中，我们提出了一种名为无调优反演增强控制（TIC）的新方法，该方法直接将反演过程中的特征与采样过程中的特征相关联，以缓解DDIM重建中的不一致性。具体而言，我们的方法有效地从自注意力层中的键和值特征中获取反演特征，并通过这些反演特征增强采样过程，从而实现精确重建和内容一致性编辑。为了将我们方法的适用性扩展到一般编辑场景，我们还提出了一种掩码引导的注意力拼接策略，该策略结合了反演过程和朴素DDIM编辑过程的内容。实验表明，所提出的方法在重建和一致性编辑方面优于先前的工作，并在各种设置下产生了令人印象深刻的结果。
英文摘要：
Consistent editing of real images is a challenging task, as it requires performing non-rigid edits (e.g., changing postures) to the main objects in the input image without changing their identity or attributes. To guarantee consistent attributes, some existing methods fine-tune the entire model or the textual embedding for structural consistency, but they are time-consuming and fail to perform non-rigid edits. Other works are tuning-free, but their performances are weakened by the quality of Denoising Diffusion Implicit Model (DDIM) reconstruction, which often fails in real-world scenarios. In this paper, we present a novel approach called Tuning-free Inversion-enhanced Control (TIC), which directly correlates features from the inversion process with those from the sampling process to mitigate the inconsistency in DDIM reconstruction. Specifically, our method effectively obtains inversion features from the key and value features in the self-attention layers, and enhances the sampling process by these inversion features, thus achieving accurate reconstruction and content-consistent editing. To extend the applicability of our method to general editing scenarios, we also propose a mask-guided attention concatenation strategy that combines contents from both the inversion and the naive DDIM editing processes. Experiments show that the proposed method outperforms previous works in reconstruction and consistent editing, and produces impressive results in various settings.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
