VASE：真实视频的以对象为中心的外观与形状操控
VASE: Object-Centric Appearance and Shape Manipulation of Real Videos
AI总结：
本文提出以对象为中心的视频编辑框架VASE，基于预训练图像条件扩散模型并集成时间维度处理层，实现对真实视频中对象外观的精准控制及结构性形状修改，其图像驱动编辑性能达SOTA且具备新颖形状编辑能力。
摘要：
近期，受大规模文本生成图像生成模型成功的推动，多项研究探讨了视频编辑任务。然而，这些方法大多利用基础扩散模型提供的先验，通过文本对画面进行整体编辑，并侧重于提升帧间的时间一致性。在本研究中，我们提出了一个以对象为中心的框架，旨在控制对象的外观，并显著地能够对对象执行精确且明确的结构性修改。我们基于预训练的图像条件扩散模型构建该框架，集成了处理时间维度的网络层，并提出了训练策略和架构修改以实现形状控制。我们在图像驱动的视频编辑任务上评估了该方法，结果显示其性能与现有技术相当，并展示了新颖的形状编辑能力。更多细节、代码和示例可访问项目页面：https://helia95.github.io/vase-website/
英文摘要：
Recently, several works tackled the video editing task fostered by the success of large-scale text-to-image generative models. However, most of these methods holistically edit the frame using the text, exploiting the prior given by foundation diffusion models and focusing on improving the temporal consistency across frames. In this work, we introduce a framework that is object-centric and is designed to control both the object's appearance and, notably, to execute precise and explicit structural modifications on the object. We build our framework on a pre-trained image-conditioned diffusion model, integrate layers to handle the temporal dimension, and propose training strategies and architectural modifications to enable shape control. We evaluate our method on the image-driven video editing task showing similar performance to the state-of-the-art, and showcasing novel shape-editing capabilities. Further details, code and examples are available on our project page: https://helia95.github.io/vase-website/
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
