I2VEdit：基于图像到视频扩散模型的首帧引导视频编辑
I2VEdit: First-Frame-Guided Video Editing via Image-to-Video Diffusion Models
AI总结：
I2VEdit利用预训练图像到视频模型，将单帧图像编辑传播至整个视频，通过粗运动提取与外观细化处理全局、局部编辑及形状变化，实现高质量时间一致的视频编辑。
摘要：
扩散模型显著的生成能力激发了图像和视频编辑领域的广泛研究。与在时间维度上面临额外挑战的视频编辑相比，图像编辑已经见证了更多样化、高质量方法以及像Photoshop这样功能更强大的软件的发展。鉴于这一差距，我们引入了一种新颖且通用的解决方案，通过使用预训练的图像到视频模型，将编辑从单个帧传播到整个视频，从而将图像编辑工具的适用性扩展到视频。我们的方法名为I2VEdit，根据编辑的程度自适应地保留源视频的视觉和运动完整性，有效处理全局编辑、局部编辑和适度的形状变化，而现有方法无法完全实现这些。我们方法的核心是两个主要过程：粗粒度运动提取，用于将基本运动模式与原始视频对齐；以及外观细化，通过细粒度注意力匹配进行精确调整。我们还结合了跳跃间隔策略，以减轻跨多个视频片段的自回归生成导致的质量下降。实验结果表明，我们的框架在细粒度视频编辑方面具有优越性能，证明了其产生高质量、时间一致性输出的能力。
英文摘要：
The remarkable generative capabilities of diffusion models have motivated extensive research in both image and video editing. Compared to video editing which faces additional challenges in the time dimension, image editing has witnessed the development of more diverse, high-quality approaches and more capable software like Photoshop. In light of this gap, we introduce a novel and generic solution that extends the applicability of image editing tools to videos by propagating edits from a single frame to the entire video using a pre-trained image-to-video model. Our method, dubbed I2VEdit, adaptively preserves the visual and motion integrity of the source video depending on the extent of the edits, effectively handling global edits, local edits, and moderate shape changes, which existing methods cannot fully achieve. At the core of our method are two main processes: Coarse Motion Extraction to align basic motion patterns with the original video, and Appearance Refinement for precise adjustments using fine-grained attention matching. We also incorporate a skip-interval strategy to mitigate quality degradation from auto-regressive generation across multiple video clips. Experimental results demonstrate our framework's superior performance in fine-grained video editing, proving its capability to produce high-quality, temporally consistent outputs.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
