Vera: 一种用于内容保持视频编辑的分层扩散模型
Vera: A Layered Diffusion Model for Content-Preserving Video Editing
AI总结：
提出Vera分层扩散框架，通过生成编辑层和alpha遮罩与源视频合成，利用混合Transformer架构和高质量分层数据集，在保持内容的同时实现高质量编辑。
摘要：
视频扩散模型在视频生成和编辑方面取得了显著进展。然而，内容保持仍然是一个核心挑战：现有方法会重新生成每个像素，并且经常改变应该保持不变的元素，例如角色或背景场景。我们引入了Vera，一种用于内容保持视频编辑的分层扩散框架。Vera不是重新生成整个视频，而是生成一个编辑层以及一个用于与源视频合成的alpha遮罩，通过设计将创意编辑与内容保持分开。为了鼓励与源视频的连贯合成，我们将文本到视频的DiT扩展为混合Transformer（MoT）架构，每层有独立的DiT，通过联合自注意力进行交互。为了支持Vera的训练，我们进一步构建了一个高质量的分层数据集，包含准确的alpha遮罩、多样的场景和动态以及视觉效果。在我们的定量基准和人类偏好研究中，Vera在内容保持方面优于领先的开源视频编辑模型，同时在编辑质量上保持竞争力，仅使用了486K帧的分层训练数据。
英文摘要：
Video diffusion models have enabled remarkable progress in video generation and editing. However, content preservation remains a core challenge: existing methods regenerate every pixel and often alter elements that should remain unchanged, such as characters or background scenes. We introduce Vera, a layered diffusion framework for content-preserving video editing. Instead of regenerating the entire video, Vera generates an edit layer along with an alpha matte for compositing with the source video, separating creative editing from content preservation by design. To encourage coherent composition with the source video, we extend the text-to-video DiT into a Mixture-of-Transformers (MoT) architecture, with separate DiTs for each layer that interact through joint self-attention. To support the training of Vera, we further construct a high-quality layered dataset with accurate alpha mattes, diverse scenes and dynamics, and visual effects. Across our quantitative benchmark and human preference study, Vera outperforms leading open-source video editing models in content preservation while remaining competitive in edit quality, using 486K frames of layered training data.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
