We introduce a new family of video prediction models designed to support downstream control tasks. We call these models Video Occupancy models (VOCs). VOCs operate in a compact latent space, thus avoiding the need to make predictions about individual pixels. Unlike prior latent-space world models, VOCs directly predict the discounted distribution of future states in a single step, thus avoiding the need for multistep roll-outs. We show that both properties are beneficial when building predictive models of video for use in downstream control. Code is available at \href{https://github.com/manantomar/video-occupancy-models}{\texttt{github.com/manantomar/video-occupancy-models}}.
Beijing Academy of Artificial Intelligence（北京人工智能研究院）；IIIS, Tsinghua University（清华大学人工智能研究院）；Shanghai Jiao Tong University（上海交通大学）；Eastern Institute of Technology, Ningbo（宁波工程技术院）；The Chinese University of Hong Kong, Shenzhen（香港中文大学（深圳））；S-Lab, Nanyang Technological University（南洋理工大学S实验室）；ByteDance（字节跳动）；Kling, Kuaishou Technology（Kling，快手科技）；GigaAI（极佳科技）；AIR, Tsinghua University（清华大学人工智能研究院）
Generative Occupancy Fields for 3D Surface-Aware Image Synthesis
The Chinese University of Hong Kong（香港中文大学）；SenseTime（商汤科技）；Max Planck Institute for Informatics（马克斯·普朗克信息学研究所）；Nanyang Technological University（南洋理工大学）
DOME: Taming Diffusion Model into High-Fidelity Controllable Occupancy World Model
Horizon Robotics；Institute of Software Chinese Academy of Sciences（中国科学院软件研究所）；Institute of Automation Chinese Academy of Sciences（中国科学院自动化研究所）；The University of Hong Kong（香港大学）；University of Pennsylvania（宾夕法尼亚大学）；HKUST(GZ)（香港科技大学（广州））；University of Chinese Academy of Sciences（中国科学院大学）；HKUST（香港科技大学）
COME: Adding Scene-Centric Forecasting Control to Occupancy World Model
School of Vehicle and Mobility, Tsinghua University（清华大学车辆与移动系统学院）；State Key Laboratory of Intelligent Green Vehicle and Mobility（智能绿色车辆与移动系统国家重点实验室）；Kargobot Inc（Kargobot公司）；Nankai University（南开大学）
InfiniVerse: Occupancy Guided Unbounded Scene Generation for Autonomous Driving
The Chinese University of Hong Kong, Shenzhen（香港中文大学（深圳））；Nanyang Technological University（南洋理工大学）；Huawei Technologies Ltd.（华为技术有限公司）；University of New South Wales（新南威尔士大学）
$I^{2}$-World: Intra-Inter Tokenization for Efficient Dynamic 4D Scene Forecasting
National Key Laboratory of Human-Machine Hybrid Augmented Intelligence（人机混合增强智能国家级重点实验室）；Institute of Artificial Intelligence and Robotics, Xi’an Jiaotong University（人工智能与机器人学院，西安交通大学）
DualDiff+: Dual-Branch Diffusion for High-Fidelity Video Generation with Reward Guidance
College of Optical Science and Engineering, Zhejiang University（浙江大学光电科学与工程学院）；Institute of Automation, Chinese Academy of Sciences（中国科学院自动化研究所）；University of Science and Technology of China（中国科学技术大学）
Experimental study on surveillance video-based indoor occupancy measurement with occupant-centric control
Center for Intelligent and Networked Systems, Department of Automation, BNRist, Tsinghua University（清华大学自动化系智能与网络化系统中心，北京信息科学与技术国家研究中心）；Urban Mobility Lab, Massachusetts Institute of Technology（麻省理工学院城市移动实验室）；Singapore-MIT Alliance for Research and Technology Centre, Massachusetts Institute of Technology（麻省理工学院新加坡-麻省理工学院研究与技术联盟中心）
OmniNWM: Omniscient Driving Navigation World Models
Shanghai Jiao Tong University（上海交通大学）；Eastern Institute of Technology（东部技术研究所）；PhiGent；National University of Singapore（新加坡国立大学）；Tsinghua University（清华大学）；Wuhan University（武汉大学）
Scaling Up Occupancy-centric Driving Scene Generation: Dataset and Method
Shanghai Jiao Tong University（上海交通大学）；Eastern Institute of Technology（东部技术研究院）；School of Electronic Information and Electrical Engineering（电子信息与电气工程学院）；Li Auto（力汽车）；National University of Singapore（新加坡国立大学）；Tsinghua University（清华大学）；Ningbo Key Laboratory of Spatial Intelligence and Digital Derivative（宁波空间智能与数字衍生实验室）；Ningbo Institute of Digital Twin（宁波数字孪生研究院）
