UNIC：统一上下文视频编辑
UNIC: Unified In-Context Video Editing
AI总结：
UNIC提出以统一上下文方式在单一模型中处理多种视频编辑任务，通过将源视频token、带噪潜变量和条件token整合为连续序列并用DiT联合建模，结合任务感知RoPE和条件偏置解决任务混淆，在统一基准上取得优越性能并支持任务组合。
摘要：
文本到视频生成的最新进展激发了人们对生成式视频编辑任务的兴趣。以往的方法通常依赖任务特定的架构（例如额外的适配器模块）或专门的定制方案（例如DDIM反演），这限制了多种编辑条件的整合以及各类编辑任务的统一。在本文中，我们提出了统一上下文视频编辑（UNIC），这是一个简单而有效的框架，以上下文方式在单一模型中统一多种视频编辑任务。为了实现这种统一，我们将各种视频编辑任务的输入表示为三种类型的token：源视频token、带噪视频潜变量，以及根据具体编辑任务而变化的多模态条件token。基于这一形式化表示，我们的核心见解是将这三种类型整合为一个连续的token序列，并利用DiT的原生注意力操作对它们进行联合建模，从而消除了对任务特定适配器设计的需求。然而，在该框架下直接进行任务统一具有挑战性，由于不同任务之间视频长度各异且条件模态多样，会导致严重的token冲突和任务混淆。为了解决这些问题，我们引入了任务感知的RoPE以促进一致的时间位置编码，并引入条件偏置，使模型能够清晰地区分不同的编辑任务。这使得我们的方法能够通过参照源视频并在“上下文”中变化条件token来自适应地执行不同的视频编辑任务，并支持灵活的任务组合。为了验证我们的方法，我们构建了一个包含六种代表性视频编辑任务的统一视频编辑基准。结果表明，我们的统一方法在每个任务上都取得了优越的性能，并展现出涌现的任务组合能力。
英文摘要：
Recent advances in text-to-video generation have sparked interest in generative video editing tasks. Previous methods often rely on task-specific architectures (e.g., additional adapter modules) or dedicated customizations (e.g., DDIM inversion), which limit the integration of versatile editing conditions and the unification of various editing tasks. In this paper, we introduce UNified In-Context Video Editing (UNIC), a simple yet effective framework that unifies diverse video editing tasks within a single model in an in-context manner. To achieve this unification, we represent the inputs of various video editing tasks as three types of tokens: the source video tokens, the noisy video latent, and the multi-modal conditioning tokens that vary according to the specific editing task. Based on this formulation, our key insight is to integrate these three types into a single consecutive token sequence and jointly model them using the native attention operations of DiT, thereby eliminating the need for task-specific adapter designs. Nevertheless, direct task unification under this framework is challenging, leading to severe token collisions and task confusion due to the varying video lengths and diverse condition modalities across tasks. To address these, we introduce task-aware RoPE to facilitate consistent temporal positional encoding, and condition bias that enables the model to clearly differentiate different editing tasks. This allows our approach to adaptively perform different video editing tasks by referring the source video and varying condition tokens "in context", and support flexible task composition. To validate our method, we construct a unified video editing benchmark containing six representative video editing tasks. Results demonstrate that our unified approach achieves superior performance on each task and exhibits emergent task composition abilities.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
