UniTransfer：通过渐进式空间和时间步分解的视频概念迁移
UniTransfer: Video Concept Transfer via Progressive Spatial and Timestep Decomposition
AI总结：
提出UniTransfer架构，通过空间和时间步渐进分解实现可控视频概念迁移，引入双流到单流DiT架构和提示链机制，并构建动物数据集OpenAnimal，实验验证其高保真与可编辑性。
摘要：
我们提出了一种新颖的架构UniTransfer，该架构在渐进式范式中引入了空间和扩散时间步分解，实现了精确且可控的视频概念迁移。具体而言，在空间分解方面，我们将视频解耦为三个关键组件：前景主体、背景和运动流。基于这种分解公式，我们进一步引入了一种基于DiT的双流到单流架构，以支持对视频中不同组件的细粒度控制。我们还引入了一种基于随机掩码的自监督预训练策略，以增强从大规模无标签视频数据中学习分解表示的能力。受思维链推理范式的启发，我们重新审视了去噪扩散过程，并提出了一种提示链（CoP）机制来实现时间步分解。我们将去噪过程分解为三个不同粒度的阶段，并利用大语言模型（LLMs）生成阶段特定的指令，以逐步指导生成过程。我们还策划了一个以动物为中心的视频数据集OpenAnimal，以促进视频概念迁移研究的进展和基准测试。大量实验表明，我们的方法在多样化的参考图像和场景中实现了高质量且可控的视频概念迁移，在视觉保真度和可编辑性方面均超越了现有基线。网页：https://yu-shaonian.github.io/UniTransfer-Web/
英文摘要：
We propose a novel architecture UniTransfer, which introduces both spatial and diffusion timestep decomposition in a progressive paradigm, achieving precise and controllable video concept transfer. Specifically, in terms of spatial decomposition, we decouple videos into three key components: the foreground subject, the background, and the motion flow. Building upon this decomposed formulation, we further introduce a dual-to-single-stream DiT-based architecture for supporting fine-grained control over different components in the videos. We also introduce a self-supervised pretraining strategy based on random masking to enhance the decomposed representation learning from large-scale unlabeled video data. Inspired by the Chain-of-Thought reasoning paradigm, we further revisit the denoising diffusion process and propose a Chain-of-Prompt (CoP) mechanism to achieve the timestep decomposition. We decompose the denoising process into three stages of different granularity and leverage large language models (LLMs) for stage-specific instructions to guide the generation progressively. We also curate an animal-centric video dataset called OpenAnimal to facilitate the advancement and benchmarking of research in video concept transfer. Extensive experiments demonstrate that our method achieves high-quality and controllable video concept transfer across diverse reference images and scenes, surpassing existing baselines in both visual fidelity and editability. Web Page: https://yu-shaonian.github.io/UniTransfer-Web/
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
