arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2501.12909cs.CLcs.GRcs.MA
收藏

FilmAgent:面向虚拟3D空间端到端电影制作的多智能体框架

FilmAgent: A Multi-Agent Framework for End-to-End Film Automation in Virtual 3D Spaces

Harbin Institute of Technology (Shenzhen)(哈尔滨工业大学(深圳));Tsinghua University(清华大学)

Zhenran Xu, Longyue Wang, Jifang Wang, Zhouyi Li, Senbao Shi, Xue Yang, Yiyu Wang, Baotian Hu, Jun Yu, Min Zhang

AI总结:

本文提出FilmAgent,一个基于LLM的多智能体协作框架,模拟导演、编剧等角色,通过迭代反馈实现虚拟3D空间中的端到端电影自动化,在15个创意上全面优于基线,平均得分3.98/5,并超越单智能体o1。

摘要:

虚拟电影制作需要复杂的决策过程,包括剧本创作、虚拟摄影以及精确的演员定位和动作。受近期基于语言智能体社会的自动化决策进展的启发,本文介绍了FilmAgent,一个新颖的基于LLM的多智能体协作框架,用于在我们构建的3D虚拟空间中进行端到端的电影自动化。FilmAgent模拟了多种剧组角色,包括导演、编剧、演员和摄影师,并涵盖了电影制作工作流程的关键阶段:(1)创意开发,将头脑风暴的想法转化为结构化的故事大纲;(2)剧本创作,为每个场景细化对话和角色动作;(3)摄影,为每个镜头确定摄像机设置。一个智能体团队通过迭代反馈和修订进行协作,从而验证中间脚本并减少幻觉。我们在15个创意和4个关键方面评估生成的视频。人工评估显示,FilmAgent在所有方面均优于所有基线,平均得分为3.98分(满分5分),展示了多智能体协作在电影制作中的可行性。进一步分析表明,尽管FilmAgent使用较不先进的GPT-4o模型,但它超越了单智能体o1,展示了协调良好的多智能体系统的优势。最后,我们讨论了OpenAI的文本到视频模型Sora和我们的FilmAgent在电影制作中的互补优势和劣势。

英文摘要:

Virtual film production requires intricate decision-making processes, including scriptwriting, virtual cinematography, and precise actor positioning and actions. Motivated by recent advances in automated decision-making with language agent-based societies, this paper introduces FilmAgent, a novel LLM-based multi-agent collaborative framework for end-to-end film automation in our constructed 3D virtual spaces. FilmAgent simulates various crew roles, including directors, screenwriters, actors, and cinematographers, and covers key stages of a film production workflow: (1) idea development transforms brainstormed ideas into structured story outlines; (2) scriptwriting elaborates on dialogue and character actions for each scene; (3) cinematography determines the camera setups for each shot. A team of agents collaborates through iterative feedback and revisions, thereby verifying intermediate scripts and reducing hallucinations. We evaluate the generated videos on 15 ideas and 4 key aspects. Human evaluation shows that FilmAgent outperforms all baselines across all aspects and scores 3.98 out of 5 on average, showing the feasibility of multi-agent collaboration in filmmaking. Further analysis reveals that FilmAgent, despite using the less advanced GPT-4o model, surpasses the single-agent o1, showing the advantage of a well-coordinated multi-agent system. Lastly, we discuss the complementary strengths and weaknesses of OpenAI's text-to-video model Sora and our FilmAgent in filmmaking.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑