arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2506.05046cs.CV

FlowDirector: 无需训练的流引导文本到视频编辑

FlowDirector: Training-Free Flow Steering for Precise Text-to-Video Editing

AGI Lab, Westlake University(西湖大学AGI实验室);Central South University(中南大学);Shanghai Jiao Tong University(上海交通大学)

Guangzhao Li, Yanming Yang, Chenxi Song, Chi Zhang

AI总结:

FlowDirector通过直接在数据空间中建模编辑过程,无需训练和倒置步骤,实现了更精确的文本到视频编辑。

摘要:

文本驱动的视频编辑旨在根据自然语言指令修改视频内容。尽管最近的无需训练的方法已经利用了预训练的扩散模型,但它们往往依赖于一个倒置编辑范式。该范式在编辑前将视频映射到潜在空间。然而,倒置过程并不完全准确,常常会损害外观保真度和运动一致性。为了解决这个问题,我们引入了FlowDirector,一种新的无需训练和倒置的视频编辑框架。我们的框架将编辑过程建模为数据空间中的直接演变。它通过常微分方程(ODE)引导视频沿着其固有的时空流形平稳过渡,从而避免不准确的倒置步骤。在此基础上,我们引入了三种流修正策略,用于外观、运动和稳定性:1)方向感知的流修正放大与源方向相反的组件并移除无关项,打破保守的流线,使结构和纹理的变化更加强大。2)运动-外观解耦在每个时间步优化运动一致性作为能量项,显著提高一致性和运动转移。3)微分平均引导策略利用多个候选流之间的差异来以低成本近似低方差区域,抑制伪影并稳定轨迹。在各种编辑任务和基准上的广泛实验表明,FlowDirector在指令遵循、时间一致性和背景保留方面实现了最先进的性能,建立了无需倒置的高效视频编辑新范式。

英文摘要:

Text-driven video editing aims to modify video content based on natural language instructions. While recent training-free methods have leveraged pretrained diffusion models, they often rely on an inversion-editing paradigm. This paradigm maps the video to a latent space before editing. However, the inversion process is not perfectly accurate, often compromising appearance fidelity and motion consistency. To address this, we introduce FlowDirector, a novel training-free and inversion-free video editing framework. Our framework models the editing process as a direct evolution in the data space. It guides the video to transition smoothly along its inherent spatio-temporal manifold using an ordinary differential equation (ODE), thereby avoiding the inaccurate inversion step. From this foundation, we introduce three flow correction strategies for appearance, motion, and stability: 1) Direction-aware flow correction amplifies components that oppose the source direction and removes irrelevant terms, breaking conservative streamlines and enabling stronger structural and textural changes. 2) Motion-appearance decoupling optimizes motion agreement as an energy term at each timestep, significantly improving consistency and motion transfer. 3) Differential averaging guidance strategy leverages differences among multiple candidate flows to approximate a low variance regime at low cost, suppressing artifacts and stabilizing the trajectory. Extensive experiments across various editing tasks and benchmarks demonstrate that FlowDirector achieves state-of-the-art performance in instruction following, temporal consistency, and background preservation, establishing an efficient new paradigm for coherent video editing without inversion.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑