StableV2V: 视频到视频编辑中的形状一致性稳定
StableV2V: Stablizing Shape Consistency in Video-to-Video Editing
AI总结：
StableV2V通过分解编辑流程并建立运动与提示的对齐，提升视频到视频编辑的形状一致性与效率。
摘要：
生成式人工智能的最新进展显著促进了内容创作和编辑，其中现有研究进一步将这一激动人心的进展扩展到视频编辑领域。在此过程中，这些研究主要将源视频中的固有运动模式转移到编辑后的视频中，但通常会观察到与用户提示不一致的结果，这是因为交付的运动与编辑内容之间缺乏特定的对齐。为了解决这一限制，我们提出了一种形状一致的视频编辑方法，即StableV2V。我们的方法将整个编辑流程分解为几个顺序步骤，首先编辑第一个视频帧，然后在交付的运动和用户提示之间建立对齐，最终基于这种对齐将编辑后的内容传播到所有其他帧。此外，我们整理了一个测试基准，即DAVIS-Edit，用于对视频编辑进行全面评估，考虑了各种类型的提示和困难。实验结果和分析展示了我们的方法在性能、视觉一致性和推理效率方面优于现有最先进研究。
英文摘要：
Recent advancements of generative AI have significantly promoted content creation and editing, where prevailing studies further extend this exciting progress to video editing. In doing so, these studies mainly transfer the inherent motion patterns from the source videos to the edited ones, where results with inferior consistency to user prompts are often observed, due to the lack of particular alignments between the delivered motions and edited contents. To address this limitation, we present a shape-consistent video editing method, namely StableV2V, in this paper. Our method decomposes the entire editing pipeline into several sequential procedures, where it edits the first video frame, then establishes an alignment between the delivered motions and user prompts, and eventually propagates the edited contents to all other frames based on such alignment. Furthermore, we curate a testing benchmark, namely DAVIS-Edit, for a comprehensive evaluation of video editing, considering various types of prompts and difficulties. Experimental results and analyses illustrate the outperforming performance, visual consistency, and inference efficiency of our method compared to existing state-of-the-art studies.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
