Autonomous driving requires forecasting both geometry and semantics over time to effectively reason about future environment states. Existing vision-based occupancy forecasting methods focus on motion-related categories such as static and dynamic objects, while semantic information remains largely absent. Recent semantic occupancy forecasting approaches address this gap but rely on past occupancy predictions obtained from separate networks. This makes current methods sensitive to error accumulation and prevents learning spatio-temporal features directly from images. In this work, we present ForecastOcc, the first framework for vision-based semantic occupancy forecasting that jointly predicts future occupancy states and semantic categories. Our framework yields semantic occupancy forecasts for multiple horizons directly from past camera images, without relying on externally estimated maps. We evaluate ForecastOcc in two complementary settings: multi-view forecasting on the Occ3D-nuScenes dataset and monocular forecasting on SemanticKITTI, where we establish the first benchmark for this task. We introduce the first baselines by adapting two 2D forecasting modules within our framework. Importantly, we propose a novel architecture that incorporates a temporal cross-attention forecasting module, a 2D-to-3D view transformer, a 3D encoder for occupancy prediction, and a semantic occupancy head for voxel-level forecasts across multiple horizons. Extensive experiments on both datasets show that ForecastOcc consistently outperforms baselines, yielding semantically rich, future-aware predictions that capture scene dynamics and semantics critical for autonomous driving.
Uber ATG（优步先进技术集团）；University of Toronto（多伦多大学）；Cornell University（康奈尔大学）
In this paper, we address the important problem in self-driving of forecasting multi-pedestrian motion and their shared scene occupancy map, critical for safe navigation. Our contributions are two-fold. First, we advocate for predicting both the individual motions as well as the scene occupancy map in order to effectively deal with missing detections caused by postprocessing, e.g., confidence thresholding and non-maximum suppression. Second, we propose a Scene-Actor Graph Neural Network (SA-GNN) which preserves the relative spatial information of pedestrians via 2D convolution, and captures th…
We propose Occupancy Flow Fields, a new representation for motion forecasting of multiple agents, an important task in autonomous driving. Our representation is a spatio-temporal grid with each grid cell containing both the probability of the cell being occupied by any agent, and a two-dimensional flow vector representing the direction and magnitude of the motion in that cell. Our method successfully mitigates shortcomings of the two most commonly-used representations for motion forecasting: trajectory sets and occupancy grids. Although occupancy grids efficiently represent the probabilistic l…
Motion planning for safe autonomous driving requires learning how the environment around an ego-vehicle evolves with time. Ego-centric perception of driveable regions in a scene not only changes with the motion of actors in the environment, but also with the movement of the ego-vehicle itself. Self-supervised representations proposed for large-scale planning, such as ego-centric freespace, confound these two motions, making the representation difficult to use for downstream motion planners. In this paper, we use geometric occupancy as a natural alternative to view-dependent representations suc…
FSF-Net: Enhance 4D Occupancy Forecasting with Coarse BEV Scene Flow for Autonomous Driving
School of Electronic Information and Communications, Huazhong University of Science and Technology（华中科技大学电子信息与通信学院）；School of Electrical and Information Engineering, Tianjin University（天津大学电气自动化与信息工程学院）
Cam4DOcc: Benchmark for Camera-Only 4D Occupancy Forecasting in Autonomous Driving Applications
Shanghai Jiao Tong University（上海交通大学）；HAOMO.AI Technology Co., Ltd.（HAOMO.AI技术有限公司）；National University of Defense Technology（国防科技大学）；Beijing Institute of Technology（北京理工大学）
We introduce Hyper-Conditioned Neural Autoregressive Flow (HCNAF); a powerful universal distribution approximator designed to model arbitrarily complex conditional probability density functions. HCNAF consists of a neural-net based conditional autoregressive flow (AF) and a hyper-network that can take large conditions in non-autoregressive fashion and outputs the network parameters of the AF. Like other flow models, HCNAF performs exact likelihood inference. We conduct a number of density estimation tasks on toy experiments and MNIST to demonstrate the effectiveness and attributes of HCNAF, in…
L3S Research Center, Leibniz University Hannover（莱布尼茨汉诺威大学L3S研究中心）；Data Science & Intelligent Systems (DSIS) Research Group, University of Bonn（波恩大学数据科学与智能系统（DSIS）研究组）
With an increasing number of electric vehicles, the accurate forecasting of charging station occupation is crucial to enable reliable vehicle charging. This paper introduces a novel Deep Fusion of Dynamic and Static Information model (DFDS) to effectively forecast the charging station occupation. We exploit static information, such as the mean occupation concerning the time of day, to learn the specific charging station patterns. We supplement such static data with dynamic information reflecting the preceding charging station occupation and temporal information such as daytime and weekday. Our…
We introduce a motion forecasting (behavior prediction) method that meets the latency requirements for autonomous driving in dense urban environments without sacrificing accuracy. A whole-scene sparse input representation allows StopNet to scale to predicting trajectories for hundreds of road agents with reliable latency. In addition to predicting trajectories, our scene encoder lends itself to predicting whole-scene probabilistic occupancy grids, a complementary output representation suitable for busy urban environments. Occupancy grids allow the AV to reason collectively about the behavior o…
University of Grenoble Alpes（格勒诺布尔阿尔卑斯大学）；Inria（法国国家信息与自动化研究所）；Univ Lyon（里昂大学）；INSA Lyon（里昂国立应用科学学院）
Reliably predicting future occupancy of highly dynamic urban environments is an important precursor for safe autonomous navigation. Common challenges in the prediction include forecasting the relative position of other vehicles, modelling the dynamics of vehicles subjected to different traffic conditions, and vanishing surrounding objects. To tackle these challenges, we propose a spatio-temporal prediction network pipeline that takes the past information from the environment and semantic labels separately for generating future occupancy predictions. Compared to the current SOTA, our approach p…
RWTH Aachen University（亚琛工业大学）；Vrije Universiteit Amsterdam（阿姆斯特丹自由大学）；University of Jyväskylä（于韦斯屈莱大学）；University of Wuppertal（伍珀塔尔大学）；Fraunhofer Institute for Applied Information Technology FIT（弗劳恩霍夫应用信息技术研究所（FIT））
Smart cities around the world have begun monitoring parking areas in order to estimate available parking spots and help drivers looking for parking. The current results are promising, indeed. However, existing approaches are limited by the high cost of sensors that need to be installed throughout the city in order to achieve an accurate estimation. This work investigates the extension of estimating parking information from areas equipped with sensors to areas where they are missing. To this end, the similarity between city neighborhoods is determined based on background data, i.e., from geogra…
SuperQuadricOcc: Real-Time Self-Supervised Semantic Occupancy Estimation with Superquadric Volume Rendering
Data Driven Computer Engineering (D²iCE) Research Centre（数据驱动计算机工程（D²iCE）研究中心）；University of Limerick（利默里克大学）；Taighde Éireann – Research Ireland（爱尔兰研究——塔吉德）
OWMDrive: Causality-Aware End-to-End Autonomous Driving via 4D Occupancy World Model
The School of Artificial Intelligence, University of Chinese Academy of Sciences（中国科学院大学人工智能学院）；Waytous Inc.（Waytous公司）；The State Key Laboratory of Multimodal Artificial Intelligence Systems, Institute of Automation, Chinese Academy of Sciences（中国科学院自动化研究所多模态人工智能系统国家重点实验室）；The College of Surveying and Geo-Informatics, Tongji University（同济大学测绘与地理信息学院）
Universidade Federal de Minas Gerais（米纳斯吉拉斯联邦大学）；Federal University of Minas Gerais（米纳斯吉拉斯联邦大学）；Federal Institute of Education Science and Technology of Northern Minas Gerais（北米纳斯吉拉斯联邦教育科学技术学院）
The prediction of residential power usage is essential in assisting a smart grid to manage and preserve energy to ensure efficient use. An accurate energy forecasting at the customer level will reflect directly into efficiency improvements across the power grid system, however forecasting building energy use is a complex task due to many influencing factors, such as meteorological and occupancy patterns. In addiction, high-dimensional time series increasingly arise in the Internet of Energy (IoE), given the emergence of multi-sensor environments and the two way communication between energy con…
Forecasting mortality associated emergency department crowding
Faculty of Medicine and Health Technology, Tampere University（坦佩雷大学医学与健康技术学院）；Emergency Department, Tampere University Hospital（坦佩雷大学医院急诊科）；Kanta-Häme Central Hospital（坎塔-海梅中心医院）；Faculty of Information Technology and Communication Sciences, Tampere University（坦佩雷大学信息技术与传播科学学院）；Centre for Vascular Surgery and Interventional Radiology, Tampere University Hospital（坦佩雷大学医院血管外科与介入放射中心）
HiMAP: History-aware Map-occupancy Prediction with Fallback
Institute of Cartography and Geoinformatics, Leibniz University Hannover（莱比锡大学汉诺威分校制图与地理信息学研究所）；Institute of Information Processing, Leibniz University Hannover（莱比锡大学汉诺威分校信息处理研究所）；Faculty of Geo-Information Science and Earth Observation, University of Twente（埃因霍温大学地球信息科学与地球观测系）
InfraWorld: Motion-Aware World Modeling for Roadside 4D Occupancy Forecasting
College of Information Science and Electronic Engineering, Zhejiang University（浙江大学信息科学与电子工程学院）；School of Mechanical and Aerospace Engineering, Nanyang Technological University（南洋理工大学机械与航天工程学院）
Integrating the Expected Future in Load Forecasts with Contextually Enhanced Transformer Models
Intelligent Maintenance and Operations Systems (IMOS)（智能维护与运营系统）；École Polytechnique Fédérale de Lausanne (EPFL)（瑞士联邦理工学院洛桑分校）；Department of Mechanical and Process Engineering（机械与过程工程系）；Eidgenössische Technische Hochschule (ETH)（瑞士联邦理工学院（ETH））；Department of Civil and Environmental Engineering（土木与环境工程系）；Massachusetts Institute of Technology (MIT)（麻省理工学院）
Easy3D-Labels: Supervising Semantic Occupancy Estimation with 3D Pseudo-Labels for Automotive Perception
Department of Electronic and Computer Engineering, University of Limerick（利默里克大学电子与计算机工程系）；Data Driven Computer Engineering Research Centre, University of Limerick（利默里克大学数据驱动计算机工程研究中心）；SFI CRT Foundations in Data Science, University of Limerick（爱尔兰科学基金会数据科学基础研究中心，利默里克大学）
COME: Adding Scene-Centric Forecasting Control to Occupancy World Model
School of Vehicle and Mobility, Tsinghua University（清华大学车辆与移动系统学院）；State Key Laboratory of Intelligent Green Vehicle and Mobility（智能绿色车辆与移动系统国家重点实验室）；Kargobot Inc（Kargobot公司）；Nankai University（南开大学）
