EchoShot：多镜头肖像视频生成
EchoShot: Multi-Shot Portrait Video Generation
AI总结：
EchoShot提出原生多镜头肖像视频生成框架，通过镜头感知位置嵌入和PortraitGala数据集，实现身份一致与属性可控，并支持长视频合成。
摘要：
视频扩散模型凭借高质量的肖像视频生成能力，显著提升了艺术工作流程的生产力。然而，现有主流流程主要局限于单镜头创作，而实际应用则要求多镜头生成，并需具备身份一致性和灵活的内容可控性。在本工作中，我们提出了EchoShot，一个基于基础视频扩散模型的原生且可扩展的多镜头肖像定制框架。首先，我们在视频扩散Transformer架构中引入了镜头感知的位置嵌入机制，以建模镜头间变化，并建立多镜头视觉内容与其文本描述之间的精细对应关系。这一简单而有效的设计使得模型能够直接在多镜头视频数据上训练，且不增加额外的计算开销。为促进多镜头场景下的模型训练，我们构建了PortraitGala，一个大规模、高保真的人类中心视频数据集，具有跨镜头身份一致性和细粒度描述，如面部属性、服装和动态动作。为进一步增强适用性，我们将EchoShot扩展至基于参考图像的个人化多镜头生成，以及具有无限镜头数量的长视频合成。大量评估表明，EchoShot在多镜头肖像视频生成中实现了卓越的身份一致性以及属性级可控性。值得注意的是，所提出的框架展现了作为通用多镜头视频建模基础范式的潜力。
英文摘要：
Video diffusion models substantially boost the productivity of artistic workflows with high-quality portrait video generative capacity. However, prevailing pipelines are primarily constrained to single-shot creation, while real-world applications urge for multiple shots with identity consistency and flexible content controllability. In this work, we propose EchoShot, a native and scalable multi-shot framework for portrait customization built upon a foundation video diffusion model. To start with, we propose shot-aware position embedding mechanisms within video diffusion transformer architecture to model inter-shot variations and establish intricate correspondence between multi-shot visual content and their textual descriptions. This simple yet effective design enables direct training on multi-shot video data without introducing additional computational overhead. To facilitate model training within multi-shot scenario, we construct PortraitGala, a large-scale and high-fidelity human-centric video dataset featuring cross-shot identity consistency and fine-grained captions such as facial attributes, outfits, and dynamic motions. To further enhance applicability, we extend EchoShot to perform reference image-based personalized multi-shot generation and long video synthesis with infinite shot counts. Extensive evaluations demonstrate that EchoShot achieves superior identity consistency as well as attribute-level controllability in multi-shot portrait video generation. Notably, the proposed framework demonstrates potential as a foundational paradigm for general multi-shot video modeling.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
