通过分割后再融合的分层视频合成
Layer-Aware Video Composition via Split-then-Merge
AI总结：
StM通过分割未标注视频并融合动态前景与背景层，提升生成视频合成的控制能力和数据效率。
摘要：
我们提出了Split-then-Merge（StM），一种新的框架，旨在增强生成视频合成中的控制能力，并解决其数据稀缺问题。不同于传统方法依赖标注数据集或手工规则，StM将大量未标注视频分割成动态前景和背景层，然后自行合成这些层以学习动态主体如何与多样化场景互动。这一过程使模型能够学习生成真实视频所需的复杂合成动态。StM引入了一种新的转换感知训练流程，利用多层融合和增强来实现功能感知的合成，同时引入一种身份保持损失，在融合过程中保持前景的保真度。实验表明，StM在定量评估和人类/VLLM基于的定性评估中均优于当前最佳方法。更多细节请访问我们的项目页面：https://split-then-merge.github.io
英文摘要：
We present Split-then-Merge (StM), a novel framework designed to enhance control in generative video composition and address its data scarcity problem. Unlike conventional methods relying on annotated datasets or handcrafted rules, StM splits a large corpus of unlabeled videos into dynamic foreground and background layers, then self-composes them to learn how dynamic subjects interact with diverse scenes. This process enables the model to learn the complex compositional dynamics required for realistic video generation. StM introduces a novel transformation-aware training pipeline that utilizes a multi-layer fusion and augmentation to achieve affordance-aware composition, alongside an identity-preservation loss that maintains foreground fidelity during blending. Experiments show StM outperforms SoTA methods in both quantitative benchmarks and in humans/VLM-based qualitative evaluations. More details are available at our project page: https://split-then-merge.github.io
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
