DreamCinema：具有自由相机与3D角色的电影化迁移
DreamCinema: Cinematic Transfer with Free Camera and 3D Character
AI总结：
本文提出DreamCinema框架，将3D电影创作分解为3D角色、驱动运动、相机运动和环境四个元素，通过结构引导动画、形状感知相机优化与环境感知细化，实现用户友好的高质量电影生成。
摘要：
我们正生活在一个数字媒体蓬勃发展的时代，每个人都有潜力成为个人电影创作者。当前关于视频生成的研究表明，利用Diffusion模型在像素空间中进行可控电影创作是一条有前景的途径。然而，对过于冗长提示词的依赖以及对电影化元素（如相机运动）关注不足，导致生成的视频缺乏电影质感。此外，3D建模的缺失往往导致视频生成失败，例如不同帧中角色模型不一致，最终阻碍了观众的沉浸式体验。在本文中，我们提出了一个新的电影创作框架DreamCinema，该框架旨在利用生成模型实现用户友好的、基于3D空间的电影创作。具体而言，我们将3D电影创作分解为四个关键元素：3D角色、驱动运动、相机运动和环境。我们从用户指定的电影镜头中提取后三个元素，并基于提供的图像使用生成模型生成3D角色。为了无缝重组这些元素并确保流畅的电影创作，我们提出了结构引导的角色动画、形状感知的相机运动优化以及环境感知的生成式细化。大量实验证明了我们的方法在生成具有自由相机和3D角色的高质量电影方面的有效性。
英文摘要：
We are living in a flourishing era of digital media, where everyone has the potential to become a personal filmmaker. Current research on video generation suggests a promising avenue for controllable film creation in pixel space using Diffusion models. However, the reliance on overly verbose prompts and insufficient focus on cinematic elements (e.g., camera movement) results in videos that lack cinematic quality. Furthermore, the absence of 3D modeling often leads to failures in video generation, such as inconsistent character models at different frames, ultimately hindering the immersive experience for viewers. In this paper, we propose a new framework for film creation, Dream-Cinema, which is designed for user-friendly, 3D space-based film creation with generative models. Specifically, we decompose 3D film creation into four key elements: 3D character, driven motion, camera movement, and environment. We extract the latter three elements from user-specified film shots and generate the 3D character using a generative model based on a provided image. To seamlessly recombine these elements and ensure smooth film creation, we propose structure-guided character animation, shape-aware camera movement optimization, and environment-aware generative refinement. Extensive experiments demonstrate the effectiveness of our method in generating high-quality films with free camera and 3D characters.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
