VIA：用于全局和局部视频编辑的统一时空视频适配框架
VIA: Unified Spatiotemporal Video Adaptation Framework for Global and Local Video Editing
AI总结：
本文提出统一时空视频适配框架VIA，通过测试时编辑适配和时空适配实现全局与局部一致的分钟级长视频编辑，实验证明其编辑更忠实、连贯且局部控制更精确。
摘要：
视频编辑是数字媒体的基础支柱，涵盖娱乐、教育和专业交流等应用。然而，以往的方法往往忽视了全面理解全局和局部上下文的必要性，导致在时空维度上，尤其是对于长视频，编辑结果不准确且不一致。在本文中，我们引入了VIA，一个用于全局和局部视频编辑的统一时空视频适配框架，推动了分钟级视频一致编辑的极限。首先，为确保单帧内的局部一致性，我们设计了测试时编辑适配，以适配预训练的图像编辑模型，提高潜在编辑方向与文本指令之间的一致性，并适配掩蔽的潜在变量以实现精确的局部控制。此外，为保持整个视频序列的全局一致性，我们引入了时空适配，递归地在关键帧中收集一致的注意力变量，并策略性地将其应用于整个序列以实现编辑效果。大量实验表明，与基线方法相比，我们的VIA方法产生的编辑更忠实于源视频，在时空上下文中更连贯，局部控制更精确。更重要的是，我们展示了VIA能够在几分钟内实现一致的长视频编辑，为长视频序列上的高级视频编辑任务释放了潜力。
英文摘要：
Video editing serves as a fundamental pillar of digital media, spanning applications in entertainment, education, and professional communication. However, previous methods often overlook the necessity of comprehensively understanding both global and local contexts, leading to inaccurate and inconsistent edits in the spatiotemporal dimension, especially for long videos. In this paper, we introduce VIA, a unified spatiotemporal Video Adaptation framework for global and local video editing, pushing the limits of consistently editing minute-long videos. First, to ensure local consistency within individual frames, we designed test-time editing adaptation to adapt a pre-trained image editing model for improving consistency between potential editing directions and the text instruction, and adapts masked latent variables for precise local control. Furthermore, to maintain global consistency over the video sequence, we introduce spatiotemporal adaptation that recursively gather consistent attention variables in key frames and strategically applies them across the whole sequence to realize the editing effects. Extensive experiments demonstrate that, compared to baseline methods, our VIA approach produces edits that are more faithful to the source videos, more coherent in the spatiotemporal context, and more precise in local control. More importantly, we show that VIA can achieve consistent long video editing in minutes, unlocking the potential for advanced video editing tasks over long video sequences.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
