arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2610.15522cs.ROcs.CV
收藏

GNFormer:用于高效基于相机的占用预测的高斯神经场变换器

GNFormer: Gaussian Neural Field Transformer for Efficient Camera-based Occupancy Prediction

Southern University of Science and Technology(南方科技大学)

Guangcheng Chen, Li He, Hong Zhang

AI总结:

该研究针对现有高斯基方法的局限,提出GNF表示与GNFormer方法,在三个基准测试中实现最优IoU、mIoU及最快推理,且所需高斯基元数量远少于现有方法。

摘要:

基于相机的占用预测为机器人感知提供几何和语义场景理解,已应用于自动驾驶车辆和无人机感知系统。高斯基元是一种有前景的替代方案,它可仅表示占用区域,从而将计算和表示能力集中在需要的地方,减少空白空间的计算和内存浪费。然而,现有基于高斯的方法存在基元利用率低的问题,无效的训练目标会使一些高斯分布在空白空间,而高斯函数的有限表达能力限制了其对复杂形状的建模能力。为解决这些局限,我们提出一种新型3D表示——高斯神经场(GNF),它可准确将体素位置映射到语义占用率。在GNF中,高斯和神经网络共同参数化该场,其中高斯定义其空间支撑并为神经解码提供特征,而神经网络实现了比单独高斯函数更具表达力的占用建模。我们进一步开发了高斯神经场变换器(GNFormer),这是一种精简且灵活的方法,用于基于相机的占用预测。在Occ-ScanNet、SSCBench-KITTI-360和nuScenes-SurroundOcc基准测试中,GNFormer在对比的单帧方法中实现了最高的IoU和mIoU,以及最快的推理速度。实验中,GNFormer分别仅用256个和1024个高斯即可表示室内和室外场景,比现有基于高斯的方法所需的基元数量大幅减少。代码将发布。

英文摘要:

Camera-based occupancy prediction provides geometric and semantic scene understanding for robot perception and has been adopted in autonomous vehicle and UAV perception systems. Gaussian primitives offer a promising alternative by potentially representing only occupied regions, thereby concentrating computation and representational capacity where needed and reducing wasted computation and memory in empty space. However, existing Gaussian-based methods suffer from poor primitive utilization, as ineffective training objectives leave some Gaussians in empty space, while the limited expressiveness of Gaussian functions restricts their ability to model complex shapes. To address these limitations, we propose a novel 3D representation, Gaussian Neural Field (GNF), which can accurately map voxel positions to semantic occupancies. In GNF, Gaussians and a neural network jointly parameterize the field, where the Gaussians define its spatial support and provide features for neural decoding, while the neural network enables more expressive occupancy modeling than Gaussian functions alone. We further develop the Gaussian Neural Field Transformer (GNFormer), a streamlined and flexible method that predicts GNFs for camera-based occupancy prediction. Across Occ-ScanNet, SSCBench-KITTI-360, and nuScenes-SurroundOcc benchmarks, GNFormer achieves the highest IoU and mIoU and the fastest inference among the compared single-frame methods. In the experiments, GNFormer can represent indoor and outdoor scenes with only 256 and 1024 Gaussians, respectively, substantially fewer primitives than prior Gaussian-based methods. The code will be released.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑