StreamGVE: 无需训练的视频编辑通过少步流式视频生成
StreamEdit: Training-Free Video Editing via Few-Step Streaming Video Generation
AI总结:
本文提出StreamGVE,一种基于噪声到数据视角的视频编辑方法,通过引入双分支快速采样和自注意力桥接以及交叉注意力接地/增强,实现了高效的视频编辑,能够在少步设置中优于现有方法。
摘要:
尽管现有的视频编辑方法通常可行,但它们往往需要许多昂贵的迭代,并且仍然难以交付高质量且令人满意的编辑结果。我们归因于普遍的数据到数据范式,这种范式不如噪声到数据生成与现代生成模型兼容。为了解决这一差距,我们重新审视视频编辑从噪声到数据的视角,并提出基于流式生成的视频编辑(StreamGVE),在保留少量步骤采样的同时无缝地注入源视频条件。基于预训练的流式生成模型,StreamGVE引入双分支快速采样,结合自注意力桥接和交叉注意力接地/增强,以满足采样和条件要求。我们进一步提出源导向的指导以提高目标生成质量,并提出视觉提示策略以增强编辑的灵活性和实用性。该方法在不同模型上均有效、稳健且具有通用性。在多样化的视频编辑任务上的广泛实验表明,StreamGVE在少步设置中也优于现有方法,即使时间成本极低。
英文摘要:
Although existing video editing methods are generally feasible, they often require many costly iterations and still struggle to deliver high-quality yet satisfying editing results. We attribute this limitation to the prevalent data-to-data paradigm, which is less compatible with modern generative models than noise-to-data generation. To address this gap, we revisit video editing from a noise-to-data perspective and propose Streaming-Generation-based Video Editing (StreamEdit), which preserves few-step sampling while seamlessly injecting source-video conditions. Built on pre-trained streaming generation models, StreamEdit introduces dual-branch fast sampling with a self-attention bridge and cross-attention grounding/boosting to satisfy both sampling and conditioning requirements. We further propose source-oriented guidance to improve target-generation quality, and a visual prompting strategy to enhance editing flexibility and practicality. The method is effective, robust, and generalizable across different models. Extensive experiments on diverse video editing tasks show that StreamEdit consistently outperforms existing approaches, even in few-step settings with minimal time cost. Code and results are available at: https://dsl-lab.github.io/StreamEdit/.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。