CamCo：相机可控的3D一致图像到视频生成
CamCo: Camera-Controllable 3D-Consistent Image-to-Video Generation
AI总结：
针对视频扩散模型缺乏精确相机位姿控制的问题，提出CamCo，通过Plücker坐标输入相机位姿并集成对极注意力模块，显著提升了图像到视频生成的3D一致性和相机控制能力。
摘要：
近期，视频扩散模型已成为普通用户可轻松使用的高质量视频内容创作的强大生成工具。然而，这些模型通常无法对视频生成的相机位姿提供精确控制，限制了电影语言的表达和用户控制。为解决这一问题，我们引入了CamCo，它允许对图像到视频生成进行细粒度的相机位姿控制。我们使用Plücker坐标为预训练的图像到视频生成器配备了准确参数化的相机位姿输入。为增强生成视频的3D一致性，我们在每个注意力块中集成了对极注意力模块，对特征图施加对极约束。此外，我们在真实世界视频上微调CamCo，这些视频的相机位姿通过运动恢复结构算法估计，以更好地合成物体运动。实验表明，与之前的模型相比，CamCo显著提升了3D一致性和相机控制能力，同时有效生成合理的物体运动。项目页面：https://ir1d.github.io/CamCo/
英文摘要：
Recently video diffusion models have emerged as expressive generative tools for high-quality video content creation readily available to general users. However, these models often do not offer precise control over camera poses for video generation, limiting the expression of cinematic language and user control. To address this issue, we introduce CamCo, which allows fine-grained Camera pose Control for image-to-video generation. We equip a pre-trained image-to-video generator with accurately parameterized camera pose input using Plücker coordinates. To enhance 3D consistency in the videos produced, we integrate an epipolar attention module in each attention block that enforces epipolar constraints to the feature maps. Additionally, we fine-tune CamCo on real-world videos with camera poses estimated through structure-from-motion algorithms to better synthesize object motion. Our experiments show that CamCo significantly improves 3D consistency and camera control capabilities compared to previous models while effectively generating plausible object motion. Project page: https://ir1d.github.io/CamCo/
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
