Videos are inherently temporal sequences by their very nature. In this work, we explore the potential of modeling videos in a chronological and scalable manner with autoregressive (AR) language models, inspired by their success in natural language processing. We introduce DiCoDe, a novel approach that leverages Diffusion-Compressed Deep Tokens to generate videos with a language model in an autoregressive manner. Unlike existing methods that employ low-level representations with limited compression rates, DiCoDe utilizes deep tokens with a considerable compression rate (a 1000x reduction in token count). This significant compression is made possible by a tokenizer trained through leveraging the prior knowledge of video diffusion models. Deep tokens enable DiCoDe to employ vanilla AR language models for video generation, akin to translating one visual "language" into another. By treating videos as temporal sequences, DiCoDe fully harnesses the capabilities of language models for autoregressive generation. DiCoDe is scalable using readily available AR architectures, and is capable of generating videos ranging from a few seconds to one minute using only 4 A100 GPUs for training. We evaluate DiCoDe both quantitatively and qualitatively, demonstrating that it performs comparably to existing methods in terms of quality while ensuring efficient training. To showcase its scalability, we release a series of DiCoDe configurations with varying parameter sizes and observe a consistent improvement in performance as the model size increases from 100M to 3B. We believe that DiCoDe's exploration in academia represents a promising initial step toward scalable video modeling with AR language models, paving the way for the development of larger and more powerful video generation models.
MegaParts: Scaling Part-Aware 3D Object Generation to 300 Parts via Token-Efficient Autoregressive Modeling
The University of Hong Kong（香港大学）；Shanghai Artificial Intelligence Laboratory（上海人工智能实验室）；Fudan University（复旦大学）；Tongji University（同济大学）；University of Science and Technology of China（中国科学技术大学）；Shanghai Jiao Tong University（上海交通大学）
Imagine Before You Draw: Visual Prompt Engineering for Image Generation
Nanyang Technological University（南洋理工大学）；National University of Singapore（国立新加坡大学）；Zhejiang University（浙江大学）；The Chinese University of Hong Kong（香港中文大学）
Weave Forcing: Compositional Memory Routing for Interactive Long Video Generation
University of Electronic Science and Technology of China (UESTC)（电子科技大学）；Mohamed bin Zayed University of Artificial Intelligence (MBZUAI)（穆罕默德·本·扎耶德人工智能大学）
Echo-Forcing: A Scene Memory Framework for Interactive Long Video Generation
State Key Laboratory of AI Safety, Institute of Computing Technology, Chinese Academy of Sciences, Beijing, China（中国科学院人工智能安全国家重点实验室，计算技术研究所，北京，中国）；University of Chinese Academy of Sciences, Beijing, China（中国科学院大学，北京，中国）；China University of Mining & Technology, Beijing（中国矿业大学（北京））；ETH Zürich（苏黎世联邦理工学院）；City College of New York, City University of New York（纽约城市学院，纽约城市大学）；Xiamen Institute of Data Intelligence, Xiamen, China（厦门数据智能研究院，厦门，中国）
Think Before You Move: Latent Motion Reasoning for Text-to-Motion Generation
Zhejiang University（浙江大学）；Hong Kong Polytechnic University（香港理工大学）；IROOTECH TECHNOLOGY（IROOTECH技术）；Wolf 1069 b Lab, Sany Group（Wolf 1069 b实验室，三一集团）；Institute of Computing Technology, Chinese Academy of Sciences（中国科学院计算技术研究所）
