基于统一潜在扩散模型的无调优保真与可编辑图像编辑
Tuning-Free Image Editing with Fidelity and Editability via Unified Latent Diffusion Model
AI总结：
本文提出无调优方法UnifyEdit，通过自注意力和交叉注意力约束及自适应时间步调度器，在统一潜在扩散框架中平衡图像编辑的结构保真度与文本可编辑性。
摘要：
在基于文本的图像编辑（TIE）中，平衡保真度与可编辑性至关重要，失败通常会导致过度编辑或编辑不足的问题。现有方法通常依赖注意力注入来保持结构，并利用预训练文本到图像（T2I）模型固有的文本对齐能力来实现可编辑性，但缺乏明确且统一的机制来恰当地平衡这两个目标。在本工作中，我们提出了UnifyEdit，这是一种无调优方法，通过执行扩散潜在优化，在统一框架内实现保真度与可编辑性的平衡集成。与直接注意力注入不同，我们开发了两种基于注意力的约束：用于结构保真度的自注意力（SA）保持约束，以及用于增强文本对齐以改善可编辑性的交叉注意力（CA）对齐约束。然而，同时施加这两种约束可能导致梯度冲突，其中一种约束占主导地位会导致过度编辑或编辑不足。为应对这一挑战，我们引入了一种自适应时间步调度器，动态调整这些约束的影响，引导扩散潜在达到最优平衡。大量定量和定性实验验证了我们方法的有效性，证明其在各种编辑任务中实现结构保持与文本对齐之间的稳健平衡方面具有优越性，优于其他最先进的方法。源代码将在 https://github.com/CUC-MIPG/UnifyEdit 提供。
英文摘要：
Balancing fidelity and editability is essential in text-based image editing (TIE), where failures commonly lead to over- or under-editing issues. Existing methods typically rely on attention injections for structure preservation and leverage the inherent text alignment capabilities of pre-trained text-to-image (T2I) models for editability, but they lack explicit and unified mechanisms to properly balance these two objectives. In this work, we introduce UnifyEdit, a tuning-free method that performs diffusion latent optimization to enable a balanced integration of fidelity and editability within a unified framework. Unlike direct attention injections, we develop two attention-based constraints: a self-attention (SA) preservation constraint for structural fidelity, and a cross-attention (CA) alignment constraint to enhance text alignment for improved editability. However, simultaneously applying both constraints can lead to gradient conflicts, where the dominance of one constraint results in over- or under-editing. To address this challenge, we introduce an adaptive time-step scheduler that dynamically adjusts the influence of these constraints, guiding the diffusion latent toward an optimal balance. Extensive quantitative and qualitative experiments validate the effectiveness of our approach, demonstrating its superiority in achieving a robust balance between structure preservation and text alignment across various editing tasks, outperforming other state-of-the-art methods. The source code will be available at https://github.com/CUC-MIPG/UnifyEdit.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
