arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2403.05131cs.AIcs.CV
收藏

Sora作为世界模型?文本到视频生成的全面调查

Sora as a World Model? A Complete Survey on Text-to-Video Generation

University of Electronic Science and Technology of China(电子科技大学);Kyung Hee University(韩国庆熙大学);The Hong Kong Polytechnic University(香港理工大学);Harbin Institute of Technology Shenzhen(哈尔滨工业大学深圳学院);Nota Inc.(Nota公司);Tongji University(同济大学)
Fachrina Dewi Puspitasari, Chaoning Zhang, Joseph Cho, Adnan Haider, Noor Ul Eman, Omer Amin, Alexis Mankowski, Muhammad Umair, Jingyao Zheng, Sheng Zheng, Lik-… 展开作者

Fachrina Dewi Puspitasari, Chaoning Zhang, Joseph Cho, Adnan Haider, Noor Ul Eman, Omer Amin, Alexis Mankowski, Muhammad Umair, Jingyao Zheng, Sheng Zheng, Lik-Hang Lee, Caiyan Qin, Tae-Ho Kim, Choong Seon Hong, Yang Yang, Heng Tao Shen

AI总结:

本文全面调查了文本到视频生成技术在世界建模中的应用,指出其在空间、行动和战略智能方面的进展,但仍需解决多样性与一致性之间的权衡问题。

摘要:

从动画MNIST到用Sora模拟世界,文本生成视频的技术发展速度惊人。本文系统讨论了文本到视频生成技术在世界建模中支持基本要求的程度。我们整理了250多篇关于基于文本的视频合成和世界建模的研究。我们观察到,最近的模型通过遵循完整性、一致性、创新性以及人类互动和控制,越来越多地支持世界建模中的空间、行动和战略智能。我们得出结论,文本到视频生成擅长于世界建模,尽管在某些方面,如多样性与一致性之间的权衡,仍需进一步解决。

英文摘要:

The evolution of video generation from text, from animating MNIST to simulating the world with Sora, has progressed at a breakneck speed. Here, we systematically discuss how far text-to-video generation technology supports essential requirements in world modeling. We curate 250+ studies on text-based video synthesis and world modeling. We then observe that recent models increasingly support spatial, action, and strategic intelligences in world modeling through adherence to completeness, consistency, invention, as well as human interaction and control. We conclude that text-to-video generation is adept at world modeling, although homework in several aspects, such as the diversity-consistency trade-offs, remains to be addressed.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑