CinePreGen：基于引擎驱动扩散的可控摄像机视频预可视化
CinePreGen: Camera Controllable Video Previsualization via Engine-powered Diffusion
AI总结：
CinePreGen提出一种结合引擎驱动扩散与多掩码IP-Adapter的视频预可视化系统，通过可控摄像机界面实现电影级运镜，降低开发复杂度并优于现有AI视频工作流。
摘要：
随着视频生成式AI模型（如SORA）的进步，创作者越来越多地使用这些技术来增强视频预可视化。然而，他们面临AI工作流不完整和不匹配的挑战。现有方法主要依赖文本描述，难以处理摄像机摆放这一预可视化的关键组成部分。为解决这些问题，我们提出了CinePreGen，一个由引擎驱动扩散增强的可视化预可视化系统。该系统具有新颖的摄像机和故事板界面，提供从全局到局部摄像机调整的动态控制。它与用户友好的AI渲染工作流相结合，旨在通过多重掩码IP-Adapter和引擎模拟准则实现一致的结果。在我们的综合评估研究中，我们证明该系统降低了开发粘度（即开发过程中的复杂性和挑战），满足了用户在设计过程中对广泛控制和迭代的需求，并在电影级摄像机运动方面优于其他AI视频制作工作流，实验结果和受试者内用户研究均证实了这一点。凭借直观的摄像机控制和逼真的摄像机运动渲染，CinePreGen在改善个人创作者和行业专业人士的视频制作方面展现出巨大潜力。
英文摘要：
With advancements in video generative AI models (e.g., SORA), creators are increasingly using these techniques to enhance video previsualization. However, they face challenges with incomplete and mismatched AI workflows. Existing methods mainly rely on text descriptions and struggle with camera placement, a key component of previsualization. To address these issues, we introduce CinePreGen, a visual previsualization system enhanced with engine-powered diffusion. It features a novel camera and storyboard interface that offers dynamic control, from global to local camera adjustments. This is combined with a user-friendly AI rendering workflow, which aims to achieve consistent results through multi-masked IP-Adapter and engine simulation guidelines. In our comprehensive evaluation study, we demonstrate that our system reduces development viscosity (i.e., the complexity and challenges in the development process), meets users' needs for extensive control and iteration in the design process, and outperforms other AI video production workflows in cinematic camera movement, as shown by our experiments and a within-subjects user study. With its intuitive camera controls and realistic rendering of camera motion, CinePreGen shows great potential for improving video production for both individual creators and industry professionals.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
