揭示流变换器中的反演与不变性用于多功能图像编辑
Unveil Inversion and Invariance in Flow Transformer for Versatile Image Editing
AI总结：
针对流变换器免调参编辑中反演不准确和不变性控制不足的问题，提出两阶段反演与基于自适应层归一化的文本特征控制，实现刚性和非刚性编辑，实验验证了灵活准确的编辑效果。
摘要：
利用流变换器的大型生成先验进行免调参图像编辑，需要真实的反演将图像投影到模型域中，以及灵活的不变性控制机制以保留非目标内容。然而，现有的扩散反演在基于流的模型中表现不足，且不变性控制无法协调多样化的刚性和非刚性编辑任务。为解决这些问题，我们系统地分析了基于流变换器的反演与不变性控制。具体而言，我们揭示欧拉反演与DDIM具有相似结构，但更容易受近似误差影响。因此，我们提出了一种两阶段反演方法，首先细化速度估计，然后补偿残余误差，使其更贴近模型先验并有利于编辑。同时，我们提出了不变性控制，通过操纵自适应层归一化中的文本特征，将文本提示的变化与图像语义联系起来。该机制能同时保留非目标内容，同时允许刚性和非刚性操作，支持多种编辑类型，如视觉文本、数量、面部表情等。在多种场景上的实验验证了我们的框架实现了灵活且准确的编辑，解锁了流变换器在多功能图像编辑中的潜力。
英文摘要：
Leveraging the large generative prior of the flow transformer for tuning-free image editing requires authentic inversion to project the image into the model's domain and a flexible invariance control mechanism to preserve non-target contents. However, the prevailing diffusion inversion performs deficiently in flow-based models, and the invariance control cannot reconcile diverse rigid and non-rigid editing tasks. To address these, we systematically analyze the \textbf{inversion and invariance} control based on the flow transformer. Specifically, we unveil that the Euler inversion shares a similar structure to DDIM yet is more susceptible to the approximation error. Thus, we propose a two-stage inversion to first refine the velocity estimation and then compensate for the leftover error, which pivots closely to the model prior and benefits editing. Meanwhile, we propose the invariance control that manipulates the text features within the adaptive layer normalization, connecting the changes in the text prompt to image semantics. This mechanism can simultaneously preserve the non-target contents while allowing rigid and non-rigid manipulation, enabling a wide range of editing types such as visual text, quantity, facial expression, etc. Experiments on versatile scenarios validate that our framework achieves flexible and accurate editing, unlocking the potential of the flow transformer for versatile image editing.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
