CamC2V: 上下文感知的可控视频生成
CamC2V: Context-aware Controllable Video Generation
AI总结：
提出CamC2V模型，通过集成多图像条件与3D约束及相机控制，实现上下文感知的连贯视频生成，在RealEstate10K数据集上FVD提升24.09%。
摘要：
近年来，图像到视频（I2V）扩散模型展示了令人印象深刻的场景理解和生成质量，通过引入图像条件来指导生成。然而，这些模型主要将静态图像动画化，而不扩展其提供的上下文。引入额外的约束，如相机轨迹，可以增强多样性，但往往会降低视觉质量，限制了它们在需要忠实场景表示的任务中的适用性。我们提出了CamC2V，一种上下文到视频（C2V）模型，它将多个图像条件作为上下文与3D约束以及相机控制集成在一起，以丰富全局语义和细粒度视觉细节。这使得视频生成更加连贯且上下文感知。此外，我们论证了有效上下文表示中时间感知的必要性。我们在RealEstate10K数据集上的全面研究表明，视觉质量和相机可控性提高了24.09%（FVD）。我们的代码公开在：https://github.com/LDenninger/CamC2V。
英文摘要：
Recently, image-to-video (I2V) diffusion models have demonstrated impressive scene understanding and generative quality, incorporating image conditions to guide generation. However, these models primarily animate static images without extending beyond their provided context. Introducing additional constraints, such as camera trajectories, can enhance diversity but often degrade visual quality, limiting their applicability for tasks requiring faithful scene representation. We propose CamC2V, a context-to-video (C2V) model that integrates multiple image conditions as context with 3D constraints alongside camera control to enrich both global semantics and fine-grained visual details. This enables more coherent and context-aware video generation. Moreover, we motivate the necessity of temporal awareness for an effective context representation. Our comprehensive study on the RealEstate10K dataset demonstrates a $24.09\%$ (FVD) improvement in visual quality and camera controllability. Our code is publicly available at: https://github.com/LDenninger/CamC2V.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
