CamViG：基于多模态Transformer的相机感知图像到视频生成
CamViG: Camera Aware Image-to-Video Generation with Multimodal Transformers
AI总结：
本文提出CamViG，通过将3D相机运动编码为调节信号，扩展多模态Transformer实现可控的图像到视频生成，并验证了相机控制的有效性与路径准确性。
摘要：
我们将多模态Transformer扩展为包含3D相机运动作为视频生成任务的调节信号。生成式视频模型正变得日益强大，因此研究工作聚焦于控制此类模型输出的方法。我们提出通过将生成的视频基于视频生成过程中三维相机运动的编码进行调节，从而为生成式视频方法添加虚拟3D相机控制。结果表明，我们能够（1）从单帧和相机信号出发，在视频生成过程中成功控制相机，并且（2）我们使用传统计算机视觉方法展示了所生成3D相机路径的准确性。
英文摘要：
We extend multimodal transformers to include 3D camera motion as a conditioning signal for the task of video generation. Generative video models are becoming increasingly powerful, thus focusing research efforts on methods of controlling the output of such models. We propose to add virtual 3D camera controls to generative video methods by conditioning generated video on an encoding of three-dimensional camera movement over the course of the generated video. Results demonstrate that we are (1) able to successfully control the camera during video generation, starting from a single frame and a camera signal, and (2) we demonstrate the accuracy of the generated 3D camera paths using traditional computer vision methods.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
