Sora as a World Model? A Complete Survey on Text-to-Video Generation
University of Electronic Science and Technology of China(电子科技大学);Kyung Hee University(韩国庆熙大学);The Hong Kong Polytechnic University(香港理工大学);Harbin Institute of Technology Shenzhen(哈尔滨工业大学深圳学院);Nota Inc.(Nota公司);Tongji University(同济大学)
Fachrina Dewi Puspitasari, Chaoning Zhang, Joseph Cho, Adnan Haider, Noor Ul Eman, Omer Amin, Alexis Mankowski, Muhammad Umair, Jingyao Zheng, Sheng Zheng, Lik-… 展开作者
Fachrina Dewi Puspitasari, Chaoning Zhang, Joseph Cho, Adnan Haider, Noor Ul Eman, Omer Amin, Alexis Mankowski, Muhammad Umair, Jingyao Zheng, Sheng Zheng, Lik-Hang Lee, Caiyan Qin, Tae-Ho Kim, Choong Seon Hong, Yang Yang, Heng Tao Shen
备注:
First complete survey on Text-to-Video Generation from World Model perspective, 35 pages
The evolution of video generation from text, from animating MNIST to simulating the world with Sora, has progressed at a breakneck speed. Here, we systematically discuss how far text-to-video generation technology supports essential requirements in world modeling. We curate 250+ studies on text-based video synthesis and world modeling. We then observe that recent models increasingly support spatial, action, and strategic intelligences in world modeling through adherence to completeness, consistency, invention, as well as human interaction and control. We conclude that text-to-video generation is adept at world modeling, although homework in several aspects, such as the diversity-consistency trade-offs, remains to be addressed.
Gender Bias in Text-to-Video Generation Models: A case study of Sora
Aligarh Muslim University(阿利加尔穆斯林大学);VIT Bhopal University(韦洛尔理工大学博帕尔分校);Nanyang Technological University(南洋理工大学);Technical University of Munich(慕尼黑工业大学);Imperial College London(帝国理工学院);Edinburgh Napier University(爱丁堡龙比亚大学)
本文通过分析 Sora 对性别中立及刻板印象提示生成的视频,揭示其将特定性别与行为和职业不当关联的偏见及训练数据根源。
Sora Generates Videos with Stunning Geometrical Consistency
Nankai University(南开大学);ByteDance Inc(字节跳动公司);Wuhan University(武汉大学);Nankai International Advanced Research Institute (Shenzhen Futian)(南开大学国际高等研究院(深圳福田))
Simple Visual Artifact Detection in Sora-Generated Videos
Joshigakuin Senior High School(女子学院高级中学);National Institute of Advanced Industrial Science and Technology (AIST)(国立先进工业科学技术研究院);Visual Geometry Group, University of Oxford(牛津大学视觉几何组)
Safe-Sora: Safe Text-to-Video Generation via Graphical Watermarking
Tsinghua University(清华大学);Institute of Automation, Chinese Academy of Sciences(中国科学院自动化研究所);South China University of Technology(华南理工大学);Guangdong Laboratory of Artificial Intelligence and Digital Economy (SZ)(广东人工智能与数字经济实验室);Singapore Management University(新加坡管理学院)
Mora: Enabling Generalist Video Generation via A Multi-Agent Framework
University of Notre Dame(圣母大学);Lehigh University(理海大学);LinkedIn Corporation(领英公司);MBZUAI(穆罕默德·本·扎耶德人工智能大学);Nanyang Technological University(南洋理工大学);Peking University(北京大学);Microsoft Research(微软研究院)
Thinking with Video: Video Generation as a Promising Multimodal Reasoning Paradigm
Institute of Trustworthy Embodied AI, Fudan University(复旦大学可信具身智能研究所);Shanghai Innovation Institute(上海创新研究院);Shanghai Key Laboratory of Multimodal Embodied AI(上海市多模态具身人工智能重点实验室);College of Computer Science and Artificial Intelligence, Fudan University(复旦大学计算机科学与人工智能学院);The Chinese University of Hong Kong(香港中文大学);Central South University(中南大学);Fudan University(复旦大学)
FAIRT2V: Training-Free Debiasing for Text-to-Video Diffusion Models
School of Computer Science and Engineering, University of New South Wales, Australia.(新南威尔士大学计算机科学与工程学院,澳大利亚);State Key Laboratory for Novel Software Technology, Nanjing University, China(南京大学新型软件技术国家重点实验室,中国)
Hardware-Friendly Static Quantization Method for Video Diffusion Transformers
Division of the Humanities and Social Sciences(人文与社会科学系);California Institute of Technology(加州理工学院);Device Solutions Research America(设备解决方案研究美国);Samsung Semiconductor Inc.(三星半导体公司)
Video-Bench: Human-Aligned Video Generation Benchmark
Shanghai Jiao Tong University(上海交通大学);Stanford University(斯坦福大学);Fellou AI(Fellou人工智能);Fudan University(复旦大学);Carnegie Mellon University(卡内基梅隆大学);Peking University(北京大学);Hong Kong Polytechnic University(香港理工大学);University of Glasgow(格拉斯哥大学);City University of Hong Kong(香港城市大学);Westlake University(西湖大学);LiveX AI(LiveX人工智能);National University of Singapore(新加坡国立大学)
Rethinking Position Embedding as a Context Controller for Multi-Reference and Multi-Shot Video Generation
Wuhan University(武汉大学);University of Science and Technology of China(中国科学技术大学);Hong Kong University of Science and Technology(香港科技大学);Tsinghua University(清华大学)