Movie Gen：一组媒体基础模型
Movie Gen: A Cast of Media Foundation Models
AI总结：
Movie Gen是一组媒体基础模型，通过30B参数Transformer等创新，在文本到视频、个性化、编辑及音频生成等任务上达到最先进水平，支持1080p高清视频与同步音频。
摘要：
我们提出了Movie Gen，一组基础模型，能够生成具有不同宽高比和同步音频的高质量1080p高清视频。我们还展示了其他能力，如基于指令的精确视频编辑和基于用户图像生成个性化视频。我们的模型在多个任务上树立了新的最先进水平：文本到视频合成、视频个性化、视频编辑、视频到音频生成以及文本到音频生成。我们最大的视频生成模型是一个30B参数的Transformer，训练时最大上下文长度为73K视频令牌，对应于以每秒16帧生成16秒的视频。我们在架构、潜在空间、训练目标和方案、数据整理、评估协议、并行化技术和推理优化方面展示了多项技术创新和简化，这些使我们能够从扩展预训练数据、模型规模和训练计算中获益，用于训练大规模媒体生成模型。我们希望这篇论文能帮助研究社区加速媒体生成模型的进展和创新。本文所有视频可在https://go.fb.me/MovieGenResearchVideos获取。
英文摘要：
We present Movie Gen, a cast of foundation models that generates high-quality, 1080p HD videos with different aspect ratios and synchronized audio. We also show additional capabilities such as precise instruction-based video editing and generation of personalized videos based on a user's image. Our models set a new state-of-the-art on multiple tasks: text-to-video synthesis, video personalization, video editing, video-to-audio generation, and text-to-audio generation. Our largest video generation model is a 30B parameter transformer trained with a maximum context length of 73K video tokens, corresponding to a generated video of 16 seconds at 16 frames-per-second. We show multiple technical innovations and simplifications on the architecture, latent spaces, training objectives and recipes, data curation, evaluation protocols, parallelization techniques, and inference optimizations that allow us to reap the benefits of scaling pre-training data, model size, and training compute for training large scale media generation models. We hope this paper helps the research community to accelerate progress and innovation in media generation models. All videos from this paper are available at https://go.fb.me/MovieGenResearchVideos.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
