调控整流流以实现反演与编辑
Taming Rectified Flow for Inversion and Editing
AI总结：
针对基于整流流的扩散Transformer反演精度不足限制编辑下游任务的问题，提出免训练采样器RF-Solver通过高阶泰勒展开提升ODE求解精度，并基于此构建特征共享编辑框架RF-Edit，无需额外训练即可在图像视频任务中实现高精度反演和保结构的高质量编辑。
摘要：
基于整流流的扩散Transformer（如FLUX和OpenSora）已在图像和视频生成领域展现出卓越性能。尽管这些模型具备强大的生成能力，但它们往往存在反演不准确的问题，这会进一步限制其在图像和视频编辑等下游任务中的效用。为解决该问题，我们提出RF-Solver，这是一种新型免训练采样器，通过缓解整流流常微分方程（ODE）求解过程中的误差，有效提升反演精度。具体而言，我们推导了整流流ODE的精确表达式，并应用高阶泰勒展开估计其非线性分量，显著提升了每个时间步的ODE求解精度。在RF-Solver的基础上，我们进一步提出RF-Edit，这是一种基于特征共享的通用图像和视频编辑框架。通过将反演过程中的自注意力特征融入编辑过程，RF-Edit在实现高质量编辑结果的同时，有效保留了源图像或视频的结构信息。我们的方法兼容任何预训练的、面向图像和视频任务的基于整流流的模型，无需额外训练或优化。在图像和视频两种模态的生成、反演及编辑任务上开展的大量实验，证明了我们方法的优越性和通用性。源代码可在https://github.com/wangjiangshan0725/RF-Solver-Edit 获取。
英文摘要：
Rectified-flow-based diffusion transformers like FLUX and OpenSora have demonstrated outstanding performance in the field of image and video generation. Despite their robust generative capabilities, these models often struggle with inversion inaccuracies, which could further limit their effectiveness in downstream tasks such as image and video editing. To address this issue, we propose RF-Solver, a novel training-free sampler that effectively enhances inversion precision by mitigating the errors in the ODE-solving process of rectified flow. Specifically, we derive the exact formulation of the rectified flow ODE and apply the high-order Taylor expansion to estimate its nonlinear components, significantly enhancing the precision of ODE solutions at each timestep. Building upon RF-Solver, we further propose RF-Edit, a general feature-sharing-based framework for image and video editing. By incorporating self-attention features from the inversion process into the editing process, RF-Edit effectively preserves the structural information of the source image or video while achieving high-quality editing results. Our approach is compatible with any pre-trained rectified-flow-based models for image and video tasks, requiring no additional training or optimization. Extensive experiments across generation, inversion, and editing tasks in both image and video modalities demonstrate the superiority and versatility of our method. The source code is available at https://github.com/wangjiangshan0725/RF-Solver-Edit.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
