CineMaster：一种用于电影级文本转视频生成的3D感知可控框架
CineMaster: A 3D-Aware and Controllable Framework for Cinematic Text-to-Video Generation
AI总结：
该研究提出CineMaster框架，通过交互式3D条件信号构建与扩散模型引导两阶段流程，结合自动化数据标注管道，实现3D感知可控的电影级文本转视频生成，性能优于现有方法。
摘要：
在这项工作中，我们提出了CineMaster，一种用于3D感知可控文本转视频生成的新型框架。我们的目标是为用户提供与专业电影导演相当的可控性：场景内物体的精确放置、3D空间中物体与相机的灵活操作，以及对渲染帧的直观布局控制。为实现这一目标，CineMaster分为两个阶段运行。第一阶段，我们设计了一个交互式工作流，允许用户通过定位物体边界框和定义3D空间内的相机运动，直观地构建3D感知条件信号。第二阶段，这些控制信号——包括渲染的深度图、相机轨迹和物体类别标签——作为文本转视频扩散模型的引导，确保生成用户预期的视频内容。此外，为克服缺乏带有3D物体运动和相机姿态标注的真实场景数据集的问题，我们精心建立了一个自动化数据标注管道，从大规模视频数据中提取3D边界框和相机轨迹。大量定性和定量实验表明，CineMaster显著优于现有方法，实现了卓越的3D感知文本转视频生成。项目页面：https://cinemaster-dev.github.io/。
英文摘要：
In this work, we present CineMaster, a novel framework for 3D-aware and controllable text-to-video generation. Our goal is to empower users with comparable controllability as professional film directors: precise placement of objects within the scene, flexible manipulation of both objects and camera in 3D space, and intuitive layout control over the rendered frames. To achieve this, CineMaster operates in two stages. In the first stage, we design an interactive workflow that allows users to intuitively construct 3D-aware conditional signals by positioning object bounding boxes and defining camera movements within the 3D space. In the second stage, these control signals--comprising rendered depth maps, camera trajectories and object class labels--serve as the guidance for a text-to-video diffusion model, ensuring to generate the user-intended video content. Furthermore, to overcome the scarcity of in-the-wild datasets with 3D object motion and camera pose annotations, we carefully establish an automated data annotation pipeline that extracts 3D bounding boxes and camera trajectories from large-scale video data. Extensive qualitative and quantitative experiments demonstrate that CineMaster significantly outperforms existing methods and implements prominent 3D-aware text-to-video generation. Project page: https://cinemaster-dev.github.io/.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
