SAMA：用于指令引导视频编辑的因子化语义锚定与运动对齐
SAMA: Factorized Semantic Anchoring and Motion Alignment for Instruction-Guided Video Editing
AI总结：
SAMA通过因子化语义锚定和运动对齐方法，在无需外部先验知识的情况下实现精确语义修改与运动保真的平衡，展示了强大的零样本视频编辑能力。
摘要：
当前指令引导视频编辑模型难以同时平衡精确的语义修改与忠实的运动保留。尽管现有方法依赖注入显式的外部先验（如VLM特征或结构条件）来缓解这些问题，但这种依赖严重限制了模型的鲁棒性和泛化能力。为克服这一限制，我们提出了SAMA（因子化语义锚定与运动对齐），一种将视频编辑因子化为语义锚定和运动建模的框架。首先，我们引入语义锚定，通过联合预测语义标记和视频潜在特征，在稀疏锚帧上建立可靠的视觉锚点，使模型能够进行纯粹的指令感知结构规划。其次，运动对齐通过在以运动为中心的视频修复预训练任务（立方体修复、速度扰动和管子洗牌）上预训练相同的主干网络，使模型能够直接从原始视频中内部化时间动态。SAMA通过两阶段流程进行优化：首先是一个因子化预训练阶段，学习内在的语义-运动表示，无需配对的视频-指令编辑数据；随后是监督微调，使用配对的编辑数据。值得注意的是，因子化预训练本身已经产生了强大的零样本视频编辑能力，验证了所提出的因子化方法。SAMA在开源模型中实现了最先进的性能，并与领先的商业系统（如Kling-Omni）具有竞争力。代码、模型和数据集将被发布。
英文摘要：
Current instruction-guided video editing models struggle to simultaneously balance precise semantic modifications with faithful motion preservation. While existing approaches rely on injecting explicit external priors (e.g., VLM features or structural conditions) to mitigate these issues, this reliance severely bottlenecks model robustness and generalization. To overcome this limitation, we present SAMA (factorized Semantic Anchoring and Motion Alignment), a framework that factorizes video editing into semantic anchoring and motion modeling. First, we introduce Semantic Anchoring, which establishes a reliable visual anchor by jointly predicting semantic tokens and video latents at sparse anchor frames, enabling purely instruction-aware structural planning. Second, Motion Alignment pre-trains the same backbone on motion-centric video restoration pretext tasks (cube inpainting, speed perturbation, and tube shuffle), enabling the model to internalize temporal dynamics directly from raw videos. SAMA is optimized with a two-stage pipeline: a factorized pre-training stage that learns inherent semantic-motion representations without paired video-instruction editing data, followed by supervised fine-tuning on paired editing data. Remarkably, the factorized pre-training alone already yields strong zero-shot video editing ability, validating the proposed factorization. SAMA achieves state-of-the-art performance among open-source models and is competitive with leading commercial systems (e.g., Kling-Omni). Code, models, and datasets will be released.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
