DreamForge: Motion-Aware Autoregressive Video Generation for Multi-View Driving Scenes
Zhejiang University（浙江大学）；Shanghai Artificial Intelligence Laboratory（上海人工智能实验室）；University of Science and Technology of China（中国科学技术大学）；Technical University of Munich（慕尼黑技术大学）
Jianbiao Mei, Tao Hu, Xuemeng Yang, Licheng Wen, Yu Yang, Tiantian Wei, Yukai Ma, Min Dou, Botian Shi, Yong Liu
Recent advances in diffusion models have improved controllable streetscape generation and supported downstream perception and planning tasks. However, challenges remain in accurately modeling driving scenes and generating long videos. To alleviate these issues, we propose DreamForge, an advanced diffusion-based autoregressive video generation model tailored for 3D-controllable long-term generation. To enhance the lane and foreground generation, we introduce perspective guidance and integrate object-wise position encoding to incorporate local 3D correlation and improve foreground object modeling. We also propose motion-aware temporal attention to capture motion cues and appearance changes in videos. By leveraging motion frames and an autoregressive generation paradigm,we can autoregressively generate long videos (over 200 frames) using a model trained in short sequences, achieving superior quality compared to the baseline in 16-frame video evaluations. Finally, we integrate our method with the realistic simulator DriveArena to provide more reliable open-loop and closed-loop evaluations for vision-based driving agents. Project Page: https://pjlab-adg.github.io/DriveArena/dreamforge.
DrivingGPT: Unifying Driving World Modeling and Planning with Multi-modal Autoregressive Transformers
Centre for Artificial Intelligence and Robotics, HKISI, CAS（中国科学院香港中文大学信息工程中心人工智能与机器人中心）；Institute of Automation, Chinese Academy of Sciences（中国科学院自动化研究所）；School of Artificial Intelligence, University of Chinese Academy of Sciences（中国科学院大学人工智能学院）
Epona: Autoregressive Diffusion World Model for Autonomous Driving
Horizon Robotics；Tsinghua University（清华大学）；Peking University（北京大学）；Nanjing University（南京大学）；The Hong Kong University of Science and Technology（香港科学与技术大学）；Nanyang Technological University（南洋理工大学）；Tencent Hunyuan（腾讯文心）
HorizonDrive: Self-Corrective Autoregressive World Model for Long-horizon Driving Simulation
Wuhan University（武汉大学）；The University of Tokyo（东京大学）；Horizon Robotics；Tsinghua University（清华大学）；University of Science and Technology of China（中国科学技术大学）；The Chinese University of Hong Kong（香港中文大学）
OmniNWM: Omniscient Driving Navigation World Models
Shanghai Jiao Tong University（上海交通大学）；Eastern Institute of Technology（东部技术研究所）；PhiGent；National University of Singapore（新加坡国立大学）；Tsinghua University（清华大学）；Wuhan University（武汉大学）
Point as Skeleton: Accumulated Point Cloud Enhanced Autoregressive Generation for Closed-Loop Autonomous Driving Simulation
Shanghai Jiao Tong University（上海交通大学）；Yinwang Intelligent Technology Co., Ltd.（银望智能科技有限公司）；Fudan University（复旦大学）；Shanghai Artificial Intelligence Laboratory（上海人工智能实验室）；University of Science and Technology of China（中国科学技术大学）；Georgia Institute of Technology（佐治亚理工学院）
Generating Multimodal Driving Scenes via Next-Scene Prediction
School of Software Engineering, XJTU（西安交通大学软件学院）；Horizon Robotics（地平线机器人）；School of Computer and Communication Sciences, EPFL（洛桑联邦理工学院计算机与通信科学学院）；University of Chinese Academy of Sciences（中国科学院大学）
AMP: Autoregressive Motion Prediction Revisited with Next Token Prediction for Autonomous Driving
Shanghai Jiao Tong University（上海交通大学）；Cowa Robot（科沃机器人）；Max Planck Institute for Informatics（马克斯·普朗克信息学研究所）；The Chinese University of Hong Kong Shenzhen（香港中文大学（深圳））
Efficient and Explainable End-to-End Autonomous Driving via Masked Vision-Language-Action Diffusion
Institute for Physical Artificial Intelligence (IPAI), Purdue University（物理人工智能研究所（IPAI）、普渡大学）；College of Engineering, Purdue University（工程学院、普渡大学）；Department of Computer Science, Purdue University（计算机科学系、普渡大学）
OneDrive: Unified Multi-Paradigm Driving with Vision-Language-Action Models
State Key Laboratory of Multimodal Artificial Intelligence Systems, CASIA School of Artificial Intelligence, University of Chinese Academy of Sciences AutoLab, School of Artificial Intelligence, Shanghai Jiao Tong University Voyager Research, Didi Chuxing School of Information Science（多模态人工智能系统国家重点实验室，中国科学院自动化所人工智能学院，中国科学院大学，AutoLab，上海交通大学人工智能学院，Voyager Research，滴滴出行，信息科学与技术学院）
LaGen: Towards Autoregressive LiDAR Scene Generation
University of Science and Technology of China（中国科学技术大学）；Shanghai Innovation Institute（上海创新研究院）；Shanghai Jiao Tong University（上海交通大学）；Fudan University（复旦大学）
SparseWorld: Enhancing End-to-End Autonomous Driving via World Models with Sparse Scene Representation
Institute of Cyber-Systems and Control, Zhejiang University（浙江大学控制系统研究所）；Labs, Huawei（华为2012实验室）；State Key Laboratory of Industrial Control Technology（国家工业控制技术重点实验室）
Drive-HWM: Hierarchical World Models for Dynamic-Latent Guided Autonomous Driving
School of Artificial Intelligence, Beihang University（北京航空航天大学人工智能学院）；Shanghai Jiao Tong University（上海交通大学）；Dim12 AI；GigaAI（极佳科技）；National University of Singapore（新加坡国立大学）；TeleAI
InfiniVerse: Occupancy Guided Unbounded Scene Generation for Autonomous Driving
The Chinese University of Hong Kong, Shenzhen（香港中文大学（深圳））；Nanyang Technological University（南洋理工大学）；Huawei Technologies Ltd.（华为技术有限公司）；University of New South Wales（新南威尔士大学）
NOVA: Next-step Open-Vocabulary Autoregression for 3D Multi-Object Tracking in Autonomous Driving
College of Mechanical and Vehicle Engineering, Hunan University（湖南大学机械与运载工程学院）；State Key Laboratory of Intelligent Autonomous Systems, Tongji University（同济大学智能自主系统国家重点实验室）
