Camera-based occupancy prediction provides geometric and semantic scene understanding for robot perception and has been adopted in autonomous vehicle and UAV perception systems. Gaussian primitives offer a promising alternative by potentially representing only occupied regions, thereby concentrating computation and representational capacity where needed and reducing wasted computation and memory in empty space. However, existing Gaussian-based methods suffer from poor primitive utilization, as ineffective training objectives leave some Gaussians in empty space, while the limited expressiveness of Gaussian functions restricts their ability to model complex shapes. To address these limitations, we propose a novel 3D representation, Gaussian Neural Field (GNF), which can accurately map voxel positions to semantic occupancies. In GNF, Gaussians and a neural network jointly parameterize the field, where the Gaussians define its spatial support and provide features for neural decoding, while the neural network enables more expressive occupancy modeling than Gaussian functions alone. We further develop the Gaussian Neural Field Transformer (GNFormer), a streamlined and flexible method that predicts GNFs for camera-based occupancy prediction. Across Occ-ScanNet, SSCBench-KITTI-360, and nuScenes-SurroundOcc benchmarks, GNFormer achieves the highest IoU and mIoU and the fastest inference among the compared single-frame methods. In the experiments, GNFormer can represent indoor and outdoor scenes with only 256 and 1024 Gaussians, respectively, substantially fewer primitives than prior Gaussian-based methods. The code will be released.
PanoOcc: Unified Occupancy Representation for Camera-based 3D Panoptic Segmentation
Institute of Automation, Chinese Academy of Sciences (CASIA)(中国科学院自动化研究所);School of Artificial Intelligence, University of Chinese Academy of Sciences (UCAS)(中国科学院大学人工智能学院);Centre for Artificial Intelligence and Robotics, HKISI, CAS(中国科学院香港创新研究院人工智能与机器人创新中心)
DSOcc: Leveraging Depth Awareness and Semantic Aid to Boost Camera-Based 3D Semantic Occupancy Prediction
S-Lab & College of Computing and Data Science, Nanyang Technological University(S实验室及计算与数据科学学院,南洋理工大学);MMLab, The Chinese University of Hong Kong(M实验室,香港中文大学);State Key Laboratory of Fluid Power & Mechatronic Systems, Zhejiang University(流体动力与机电系统国家重点实验室,浙江大学);SenseTime Research, Hong Kong(商汤研究,香港)
Cam4DOcc: Benchmark for Camera-Only 4D Occupancy Forecasting in Autonomous Driving Applications
Shanghai Jiao Tong University(上海交通大学);HAOMO.AI Technology Co., Ltd.(HAOMO.AI技术有限公司);National University of Defense Technology(国防科技大学);Beijing Institute of Technology(北京理工大学)
REOcc: Camera-Radar Fusion with Radar Feature Enrichment for 3D Occupancy Prediction
School of Electrical Engineering, Korea Advanced Institute of Science and Technology (KAIST)(韩国科学技术院电子工程学院);Department of Automobile and IT Convergence, Kookmin University(韩国庆熙大学汽车与IT融合系);Cho Chun Shik Graduate School of Mobility, Korea Advanced Institute of Science and Technology (KAIST)(韩国科学技术院移动技术研究生院);Robotics Program, Korea Advanced Institute of Science and Technology (KAIST)(韩国科学技术院机器人项目)
SurroundOcc: Multi-Camera 3D Occupancy Prediction for Autonomous Driving
Beijing National Research Center for Information Science and Technology(北京信息科学与技术国家研究中心);Tsinghua University(清华大学);Tianjin University(天津大学);PhiGent Robotics(鉴智机器人)
LiCROcc: Teach Radar for Accurate Semantic Occupancy Prediction using LiDAR and Camera
Institute of Cyber-Systems and Control, Zhejiang University(浙江大学 cyber 系统与控制研究所);Shanghai Artificial Intelligence Laboratory(上海人工智能实验室);Technical University of Munich(慕尼黑工业大学)
School of Computing and Augmented Intelligence, Arizona State University(亚利桑那州立大学计算与增强智能学院);Department of Computer Science and Physics, Rider University(莱德大学计算机科学与物理系)
OccTrack360: 4D Panoptic Occupancy Tracking from Surround-View Fisheye Cameras
School of Artificial Intelligence and Robotics, Hunan University(人工智能与机器人学院,湖南大学);State Key Laboratory of Extreme Photonics and Instrumentation, Zhejiang University(极端光子学与仪器国家重点实验室,浙江大学);National Engineering Research Center of Robot Visual Perception and Control Technology, Hunan University(机器人视觉感知与控制技术国家工程研究中心,湖南大学)
$M^2$-Occ: Resilient 3D Semantic Occupancy Prediction for Autonomous Driving with Incomplete Camera Inputs
School of Artificial Intelligence and Robotics and the National Engineering Research Center of Robot Visual Perception and Control Technology, Hunan University, China(湖南大学人工智能与机器人学院及机器人视觉感知与控制技术国家工程研究中心,中国);Institute for Anthropomatics and Robotics, Karlsruhe Institute of Technology, Germany(卡尔斯鲁厄理工学院人机学与机器人研究所,德国)
StereoVoxelNet: Real-Time Obstacle Detection Based on Occupancy Voxels from a Stereo Camera Using Deep Neural Networks
Institute for Experiential Robotics, Northeastern University(东北大学体验机器人研究所);Khoury College of Computer Sciences, Northeastern University(东北大学胡里计算机学院);Department of Electrical and Computer Engineering, Northeastern University(东北大学电气与计算机工程系)
障碍物检测是机器人导航中的一个安全关键问题,其中立体匹配是一种流行的基于视觉的方法。虽然深度神经网络在计算机视觉中已经显示出令人印象深刻的结果,但大多数先前的障碍物检测工作仅利用传统的立体匹配技术来满足实时的【形】反馈的计算约束。本文提出了一种计算效率高的方法,该方法采用深度神经网络直接从立体图像中检测占用。我们的方法不是从立体数据中学习点云对应关系,而是基于体积表示提取紧凑的障碍物分布。此外,我们修剪的安全无关空间的计算,在一个由粗到细的方式的基础上产生的解码器的八叉树。因此,我们在机载计算机(NVIDIA Jetson TX 2)上实现了实时的【形】性能。我们的方法可以在32米范围内准确检测障碍物,并实现更好的IoU(Intersection over Union)和CD(Chamfer Distance)分数,而计算成本仅为最先进的立体模型的2%。此外,我们验证了我们的方法的鲁棒性和现实世界的可行性,通过自主导航实验与真正的机器人。因此,我们的工作有助于缩小机器人感知中基于立体的系统与计算机视觉中最先进的立体模型之间的差距。为了应对高质量的真实室内立体数据集的稀缺性,我们使用移动机器人收集了1.36小时的立体数据集,用于微调我们的模型。有关数据集、代码和更多详细信息(包括其他可视化),请访问https://lhy.xyz/stereovoxelnet
Co-Occ: Coupling Explicit Feature Fusion with Volume Rendering Regularization for Multi-Modal 3D Semantic Occupancy Prediction
The Hong Kong University of Science and Technology (Guangzhou)(香港科技大学(广州));HKUST(GZ)(香港科技大学(广州));The Hong Kong University of Science and Technology(香港科技大学)
SPOT-Occ: Sparse Prototype-guided Transformer for Camera-based 3D Occupancy Prediction
Artificial Intelligence Thrust, The Hong Kong University of Science and Technology (Guangzhou)(香港科学与技术大学人工智能方向(广州));Department of Computer Science and Engineering, The Hong Kong University of Science and Technology(香港科学与技术大学计算机科学与工程系)
HybridOcc: NeRF Enhanced Transformer-based Multi-Camera 3D Occupancy Prediction
Academy for Engineering and Technology, Fudan University(复旦大学工程与技术学院);China FAW Group Corp., Ltd.(中国第一汽车集团有限公司);Engineering Research Center of AI and Robotics, Ministry of Education(教育部人工智能与机器人工程研究中心);Jilin Provincial Key Laboratory of Intelligence Science and Engineering(吉林省智能科学与工程重点实验室)
Doracamom: Joint 3D Detection and Occupancy Prediction with Multi-view 4D Radars and Cameras for Omnidirectional Perception
School of Automotive Studies, Tongji University(同济大学汽车学院);Momoni AI;Department of Computer Science and Engineering, The Hong Kong University of Science and Technology(香港科技大学计算机科学与工程系);Chair of Automotive Engineering, Technische Universität Berlin(柏林技术大学汽车工程学系);College of Information Science and Electronic Engineering, Zhejiang University(浙江大学信息科学与电子工程学院);School of Information and Electrical Engineering, Hangzhou City University(杭州城市学院信息与电气工程学院)
SWA-SOP: Spatially-aware Window Attention for Semantic Occupancy Prediction in Autonomous Driving
Autonomous Intelligent Systems group, Computer Science Institute VI – Intelligent Systems and Robotics, University of Bonn, Germany(bonn大学计算机科学研究所VI——智能系统与机器人组);Center for Robotics, University of Bonn, Germany(bonn大学机器人中心);Lamarr Institute for Machine Learning and Artificial Intelligence, Germany(德国lamarr机器学习与人工智能研究所)
MS-Occ: Multi-Stage LiDAR-Camera Fusion for 3D Semantic Occupancy Prediction
School of Optical-Electrical and Computer Engineering, University of Shanghai for Science and Technology(光学电子与计算机工程学院,上海科学技术大学);School of Automotive Studies, Tongji University(汽车学院,同济大学);Momoni AI;College of Science and Engineering, James Cook University(科学与工程学院,詹姆斯库克大学);School of Engineering, Swinburne University of Technology(工程学院,斯威本技术大学);School of Electrical and Electronic Engineering, Nanyang Technological University(电气与电子工程学院,南洋理工大学)
FDR-Occ: Factorized Dense Routing for Full-Spectrum 3D Occupancy Prediction
SKL-IOTSC, CIS, University of Macau(澳门大学物联网与智慧城市国家重点实验室、计算机与信息科学系);Chongqing Afari Intelligent Drive Co., Ltd.(重庆阿法里智能驾驶有限公司);Geely Automobile Research Institute (Ningbo) Co., Ltd.(吉利汽车研究院(宁波)有限公司)