arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2408.11788cs.AIcs.CLcs.CVcs.SE

DreamFactory:开创基于多智能体框架的多场景长视频生成

DreamFactory: Pioneering Multi-Scene Long Video Generation with a Multi-Agent Framework

TsingHua University(清华大学);University of Luxembourg(卢森堡大学);Beijing Institute of Technology(北京理工大学)

Zhifei Xie, Daniel Tang, Dingwei Tan, Jacques Klein, Tegawend F. Bissyand, Saad Ezzini

AI总结:

针对长多场景视频生成难题,提出基于LLM的DreamFactory多智能体框架,结合关键帧迭代与COT确保一致性,并引入跨场景评估指标及含150+视频的数据集。

摘要:

当前视频生成模型擅长创建简短逼真的片段,但在处理较长的多场景视频时存在困难。我们引入了基于LLM的框架DreamFactory来应对这一挑战。DreamFactory利用多智能体协作原则和关键帧迭代设计方法,以确保长视频的一致性和风格。它利用思维链(COT)来解决大语言模型固有的不确定性。DreamFactory能生成长篇、风格连贯且复杂的视频。评估这些长视频是一项挑战。我们提出了跨场景人脸距离分数和跨场景风格一致性分数等新指标。为促进该领域研究,我们贡献了包含超过150个人工评级视频的多场景视频数据集。

英文摘要:

Current video generation models excel at creating short, realistic clips, but struggle with longer, multi-scene videos. We introduce \texttt{DreamFactory}, an LLM-based framework that tackles this challenge. \texttt{DreamFactory} leverages multi-agent collaboration principles and a Key Frames Iteration Design Method to ensure consistency and style across long videos. It utilizes Chain of Thought (COT) to address uncertainties inherent in large language models. \texttt{DreamFactory} generates long, stylistically coherent, and complex videos. Evaluating these long-form videos presents a challenge. We propose novel metrics such as Cross-Scene Face Distance Score and Cross-Scene Style Consistency Score. To further research in this area, we contribute the Multi-Scene Videos Dataset containing over 150 human-rated videos.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑