ProEdit: 通过正确提示实现基于反向的编辑
ProEdit: Inversion-based Editing From Prompts Done Right
AI总结：
ProEdit通过改进注意力和潜在领域，实现更稳定的基于反向的编辑，能有效提升图像和视频编辑的性能和一致性。
摘要：
基于反向的视觉编辑提供了一种有效且无需训练的方式来根据用户指令编辑图像或视频。现有方法通常在采样过程中注入源图像信息以保持编辑一致性。然而，这种采样策略过度依赖源信息，这会负面影响目标图像的编辑（例如，无法按照指令改变主体的属性，如姿态、数量或颜色）。在本工作中，我们提出ProEdit来解决这一问题，既在注意力方面又在潜在方面进行改进。在注意力方面，我们引入了KV-mix，它在编辑区域混合源和目标的KV特征，从而减轻源图像对编辑区域的影响，同时保持背景一致性。在潜在方面，我们提出了Latents-Shift，它扰动源潜在的编辑区域，消除反向潜在对采样的影响。在多个图像和视频编辑基准上的广泛实验表明，我们的方法实现了SOTA性能。此外，我们的设计是即插即用的，可以无缝集成到现有的反向和编辑方法中，如RF-Solver、FireFlow和UniEdit。
英文摘要：
Inversion-based visual editing provides an effective and training-free way to edit an image or a video based on user instructions. Existing methods typically inject source image information during the sampling process to maintain editing consistency. However, this sampling strategy overly relies on source information, which negatively affects the edits in the target image (e.g., failing to change the subject's atributes like pose, number, or color as instructed). In this work, we propose ProEdit to address this issue both in the attention and the latent aspects. In the attention aspect, we introduce KV-mix, which mixes KV features of the source and the target in the edited region, mitigating the influence of the source image on the editing region while maintaining background consistency. In the latent aspect, we propose Latents-Shift, which perturbs the edited region of the source latent, eliminating the influence of the inverted latent on the sampling. Extensive experiments on several image and video editing benchmarks demonstrate that our method achieves SOTA performance. In addition, our design is plug-and-play, which can be seamlessly integrated into existing inversion and editing methods, such as RF-Solver, FireFlow and UniEdit.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
