驯服基于流的图像到视频模型以实现创意视频编辑
Taming Flow-based I2V Models for Creative Video Editing
AI总结：
本文提出免反演的IF-V2V方法，通过向量场校正、结构与运动保持初始化和偏差缓存机制，将现成基于流匹配的I2V模型高效适配用于视频编辑，实现轻量级即插即用的创意视频编辑。
摘要：
尽管图像编辑技术已经取得了显著进展，但视频编辑——旨在根据用户意图对视频进行操作——仍然是一个新兴的挑战。现有的大多数以图像为条件的视频编辑方法要么需要针对特定模型设计进行反演，要么需要大量优化，这限制了它们利用最新的图像到视频（I2V）模型将图像编辑模型的编辑能力迁移到视频领域的能力。为此，我们提出了 IF-V2V，一种免反演（Inversion-Free）方法，能够在不产生显著计算开销的情况下，将现成的基于流匹配的 I2V 模型适配用于视频编辑。为了规避反演，我们设计了带样本偏差的向量场校正（Vector Field Rectification with Sample Deviation），通过在去噪向量场中引入偏差项，将源视频的信息融入去噪过程。为了进一步以模型无关的方式确保与源视频的一致性，我们引入了结构与运动保持初始化（Structure-and-Motion-Preserving Initialization），生成具有运动感知的时间相关噪声并嵌入结构信息。我们还提出了一种偏差缓存（Deviation Caching）机制，以最小化去噪向量校正的额外计算成本，同时不会显著影响编辑质量。评估结果表明，我们的方法在编辑质量和一致性方面优于现有方法，提供了一种轻量级的即插即用解决方案来实现视觉创意。
英文摘要：
Although image editing techniques have advanced significantly, video editing, which aims to manipulate videos according to user intent, remains an emerging challenge. Most existing image-conditioned video editing methods either require inversion with model-specific design or need extensive optimization, limiting their capability of leveraging up-to-date image-to-video (I2V) models to transfer the editing capability of image editing models to the video domain. To this end, we propose IF-V2V, an Inversion-Free method that can adapt off-the-shelf flow-matching-based I2V models for video editing without significant computational overhead. To circumvent inversion, we devise Vector Field Rectification with Sample Deviation to incorporate information from the source video into the denoising process by introducing a deviation term into the denoising vector field. To further ensure consistency with the source video in a model-agnostic way, we introduce Structure-and-Motion-Preserving Initialization to generate motion-aware temporally correlated noise with structural information embedded. We also present a Deviation Caching mechanism to minimize the additional computational cost for denoising vector rectification without significantly impacting editing quality. Evaluations demonstrate that our method achieves superior editing quality and consistency over existing approaches, offering a lightweight plug-and-play solution to realize visual creativity.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
