Understanding the 3D geometry and semantics of driving scenes is critical for safe autonomous driving. Recent advances in 3D occupancy prediction have improved scene representation but often suffer from visual inconsistencies, leading to floating artifacts and poor surface localization. Existing voxel-wise losses (e.g., cross-entropy) fail to enforce visible geometric coherence. In this paper, we propose GaussRender, a module that improves 3D occupancy learning by enforcing projective consistency. Our key idea is to project both predicted and ground-truth 3D occupancy into 2D camera views, where we apply supervision. Our method penalizes 3D configurations that produce inconsistent 2D projections, thereby enforcing a more coherent 3D structure. To achieve this efficiently, we leverage differentiable rendering with Gaussian splatting. GaussRender seamlessly integrates with existing architectures while maintaining efficiency and requiring no inference-time modifications. Extensive evaluations on multiple benchmarks (SurroundOcc-nuScenes, Occ3D-nuScenes, SSCBench-KITTI360) demonstrate that GaussRender significantly improves geometric fidelity across various 3D occupancy models (TPVFormer, SurroundOcc, Symphonies), achieving state-of-the-art results, particularly on surface-sensitive metrics such as RayIoU. The code is open-sourced at https://github.com/valeoai/GaussRender.
RenderOcc: Vision-Centric 3D Occupancy Prediction with 2D Rendering Supervision
National Key Laboratory for Multimedia Information Processing（多媒体信息处理国家重点实验室）；School of CS, Peking University（北京大学计算机学院）；Xiaomi Car（小米汽车）；Nanjing University（南京大学）；Nanyang Technological University（南洋理工大学）；CUHK MMLAB（香港中文大学多媒体实验室）
SuperQuadricOcc: Real-Time Self-Supervised Semantic Occupancy Estimation with Superquadric Volume Rendering
Data Driven Computer Engineering (D²iCE) Research Centre（数据驱动计算机工程（D²iCE）研究中心）；University of Limerick（利默里克大学）；Taighde Éireann – Research Ireland（爱尔兰研究——塔吉德）
Co-Occ: Coupling Explicit Feature Fusion with Volume Rendering Regularization for Multi-Modal 3D Semantic Occupancy Prediction
The Hong Kong University of Science and Technology (Guangzhou)（香港科技大学（广州））；HKUST(GZ)（香港科技大学（广州））；The Hong Kong University of Science and Technology（香港科技大学）
UnsOcc: 3D Semantic Occupancy Prediction in Unstructured Scene via Rendering Fusion
School of Artificial Intelligence, University of Chinese Academy of Sciences（中国科学院大学人工智能学院）；Institute of Automation, Chinese Academy of Sciences（中国科学院自动化研究所）；Waytous Inc.（Waytous公司）
UniScene: Unified Occupancy-centric Driving Scene Generation
Shanghai Jiao Tong University（上海交通大学）；Ningbo Institute of Digital Twin, Eastern Institute of Technology（东方理工大学宁波数字孪生研究院）；Tsinghua University（清华大学）；MEGVII Technology（旷视科技）；Mach Drive（马赫驱动）；Fudan University（复旦大学）；University of Hong Kong（香港大学）
OG-Gaussian: Occupancy Based Street Gaussians for Autonomous Driving
School of Computer Science and Technology, University of Science and Technology of China（中国科学技术大学计算机科学与技术学院）；School of Artificial Intelligence and Data Science, University of Science and Technology of China（中国科学技术大学人工智能与数据科学学院）
Easy3D-Labels: Supervising Semantic Occupancy Estimation with 3D Pseudo-Labels for Automotive Perception
Department of Electronic and Computer Engineering, University of Limerick（利默里克大学电子与计算机工程系）；Data Driven Computer Engineering Research Centre, University of Limerick（利默里克大学数据驱动计算机工程研究中心）；SFI CRT Foundations in Data Science, University of Limerick（爱尔兰科学基金会数据科学基础研究中心，利默里克大学）
HybridOcc: NeRF Enhanced Transformer-based Multi-Camera 3D Occupancy Prediction
Academy for Engineering and Technology, Fudan University（复旦大学工程与技术学院）；China FAW Group Corp., Ltd.（中国第一汽车集团有限公司）；Engineering Research Center of AI and Robotics, Ministry of Education（教育部人工智能与机器人工程研究中心）；Jilin Provincial Key Laboratory of Intelligence Science and Engineering（吉林省智能科学与工程重点实验室）
GEOcc: Geometrically Enhanced 3D Occupancy Network with Implicit-Explicit Depth Fusion and Contextual Self-Supervision
School of Computer Science and Technology, East China Normal University（华东师范大学计算机科学与技术学院）；School of Electronics, Information and Electrical Engineering, Shanghai Jiao Tong University（上海交通大学电子信息与电气工程学院）；School of Transportation and Traffic Engineering, Central South University（中南大学交通运输工程学院）
The University of Hong Kong（香港大学）；Zhejiang University（浙江大学）；Harbin Institute of Technology（哈尔滨工业大学）；DJI（大疆创新）
Implicit neural rendering, which uses signed distance function (SDF) representation with geometric priors (such as depth or surface normal), has led to impressive progress in the surface reconstruction of large-scale scenes. However, applying this method to reconstruct a room-level scene from images may miss structures in low-intensity areas or small and thin objects. We conducted experiments on three datasets to identify limitations of the original color rendering loss and priors-embedded SDF scene representation. We found that the color rendering loss results in optimization bias against low…
MS-Occ: Multi-Stage LiDAR-Camera Fusion for 3D Semantic Occupancy Prediction
School of Optical-Electrical and Computer Engineering, University of Shanghai for Science and Technology（光学电子与计算机工程学院，上海科学技术大学）；School of Automotive Studies, Tongji University（汽车学院，同济大学）；Momoni AI；College of Science and Engineering, James Cook University（科学与工程学院，詹姆斯库克大学）；School of Engineering, Swinburne University of Technology（工程学院，斯威本技术大学）；School of Electrical and Electronic Engineering, Nanyang Technological University（电气与电子工程学院，南洋理工大学）
The Chinese University of Hong Kong（香港中文大学）；SenseTime（商汤科技）；Max Planck Institute for Informatics（马克斯·普朗克信息学研究所）；Nanyang Technological University（南洋理工大学）
The advent of generative radiance fields has significantly promoted the development of 3D-aware image synthesis. The cumulative rendering process in radiance fields makes training these generative models much easier since gradients are distributed over the entire volume, but leads to diffused object surfaces. In the meantime, compared to radiance fields occupancy representations could inherently ensure deterministic surfaces. However, if we directly apply occupancy representations to generative models, during training they will only receive sparse gradients located on object surfaces and event…
Bridging 3D Gaussians and Semantic Occupancy for Comprehensive Open-Vocabulary Scene Understanding from Unposed Images
The Hong Kong Polytechnic University（香港理工大学）；Eastern Institute of Technology（东方理工大学）；Shanghai Jiao Tong University（上海交通大学）；Wuhan University（武汉大学）；Great Wall Motor（长城汽车）；Tsinghua University（清华大学）；Ningbo Institute of Digital Twin, Eastern Institute of Technology（东方理工大学宁波数字孪生研究院）
Scaling Up Occupancy-centric Driving Scene Generation: Dataset and Method
Shanghai Jiao Tong University（上海交通大学）；Eastern Institute of Technology（东部技术研究院）；School of Electronic Information and Electrical Engineering（电子信息与电气工程学院）；Li Auto（力汽车）；National University of Singapore（新加坡国立大学）；Tsinghua University（清华大学）；Ningbo Key Laboratory of Spatial Intelligence and Digital Derivative（宁波空间智能与数字衍生实验室）；Ningbo Institute of Digital Twin（宁波数字孪生研究院）
National Cheng Kung University（国立成功大学）；National Tsing Hua University（国立清华大学）；National Taiwan University（国立台湾大学）
Geometry-aware modules are widely applied in recent deep learning architectures for scene representation and rendering. However, these modules require intrinsic camera information that might not be obtained accurately. In this paper, we propose a Spatial Transformation Routing (STR) mechanism to model the spatial properties without applying any geometric prior. The STR mechanism treats the spatial transformation as the message passing process, and the relation between the view poses and the routing weights is modeled by an end-to-end trainable neural network. Besides, an Occupancy Concept Mapp…
Korea Advanced Institute of Science and Technology（韩国科学技术院）
Single-image 3D human reconstruction aims to reconstruct the 3D textured surface of the human body given a single image. While implicit function-based methods recently achieved reasonable reconstruction performance, they still bear limitations showing degraded quality in both surface geometry and texture from an unobserved view. In response, to generate a realistic textured surface, we propose ReFu, a coarse-to-fine approach that refines the projected backside view image and fuses the refined image to predict the final human body. To suppress the diffused occupancy that causes noise in project…
EmerNeRF: Emergent Spatial-Temporal Scene Decomposition via Self-Supervision
Technion（以色列理工学院）；NVIDIA Research（英伟达研究院）；Georgia Institute of Technology（佐治亚理工学院）；University of Toronto（多伦多大学）；Stanford University（斯坦福大学）；University of Southern California（南加州大学）
Facebook AI Research（脸书人工智能研究院）；McGill University（麦吉尔大学）
Multi-view implicit scene reconstruction methods have become increasingly popular due to their ability to represent complex scene details. Recent efforts have been devoted to improving the representation of input information and to reducing the number of views required to obtain high quality reconstructions. Yet, perhaps surprisingly, the study of which views to select to maximally improve scene understanding remains largely unexplored. We propose an uncertainty-driven active vision approach for implicit scene reconstruction, which leverages occupancy uncertainty accumulated across the scene u…
VRP-UDF: Towards Unbiased Learning of Unsigned Distance Functions from Multi-view Images with Volume Rendering Priors
School of Software, Tsinghua University（清华大学软件学院）；China Telecom Wanwei Information Technology Co., Ltd.（中国电信万维信息技术有限公司）；Kuaishou Technology（快手技术）；Department of Computer Science, Wayne State University（韦恩州立大学计算机科学系）
Samsung AI Centre Toronto（三星人工智能中心（多伦多））；University of Toronto（多伦多大学）；Vector Institute for AI（向量人工智能研究院）
Differentiable rendering is an essential operation in modern vision, allowing inverse graphics approaches to 3D understanding to be utilized in modern machine learning frameworks. Explicit shape representations (voxels, point clouds, or meshes), while relatively easily rendered, often suffer from limited geometric fidelity or topological constraints. On the other hand, implicit representations (occupancy, distance, or radiance fields) preserve greater fidelity, but suffer from complex or inefficient rendering processes, limiting scalability. In this work, we endeavour to address both shortcomi…
Robotics Institute, Carnegie Mellon University（卡内基梅隆大学机器人研究所）
Recent history has seen a tremendous growth of work exploring implicit representations of geometry and radiance, popularized through Neural Radiance Fields (NeRF). Such works are fundamentally based on a (implicit) volumetric representation of occupancy, allowing them to model diverse scene structure including translucent objects and atmospheric obscurants. But because the vast majority of real-world scenes are composed of well-defined surfaces, we introduce a surface analog of such implicit models called Neural Reflectance Surfaces (NeRS). NeRS learns a neural shape representation of a closed…
