arXiv 2403.027102024-03-05cs.CVcs.RO

FastOcc：通过融合2D鸟瞰视图与透视图加速3D占用预测

FastOcc: Accelerating 3D Occupancy Prediction by Fusing the 2D Bird's-Eye View and Perspective View

School of Computer Science, Fudan University（复旦大学计算机科学技术学院）；Mogo Auto Intelligence and Telematics Information Technology Co., Ltd（摩国汽车智能与远程信息处理信息技术有限公司）；Faculty of Information Technology, Beijing University of Technology（北京工业大学信息学部）；Institute of Science and Technology for Brain-Inspired Intelligence, Fudan University（复旦大学脑智能科学与技术研究院）

Jiawei Hou, Xiaoyan Li, Wenhao Guan, Gang Zhang, Di Feng, Yuheng Du, Xiangyang Xue, Jian Pu

备注：: Accepted by ICRA 2024

阅读 PDF

arXiv 页面 ↗HTML 原文 ↗

AI总结：

针对3D占用预测推理速度不足的问题，本文提出FastOcc，用融合2D BEV与3D体素特征的类残差预测头替代3D卷积，并在Occ3D-nuScenes上实现精度与速度的先进水平。

摘要：

在自动驾驶中，3D占用预测输出体素级状态和语义标签，与3D目标检测、鸟瞰视图（BEV）语义分割等传统感知任务相比，能够更全面地理解3D场景。近期研究者广泛探索了该任务的多个方面，包括视图转换技术、真值标签生成以及精细的网络设计，旨在获得更优性能。然而，对于在自动驾驶车辆上运行至关重要的推理速度却被忽视。为此，本文提出了一种名为FastOcc的新方法。通过仔细分析输入图像分辨率、图像主干网络、视图转换和占用预测头四个部分对网络效果和延迟的影响，研究发现占用预测头在保持精度的同时，具有相当大的模型加速潜力。针对改进该组件，本文用一种新颖的类残差架构替换耗时的3D卷积网络：特征主要由轻量级2D BEV卷积网络处理，并通过整合从原始图像特征插值得到的3D体素特征进行补偿。在Occ3D-nuScenes基准上的实验表明，FastOcc以快速推理速度取得了最先进的结果。

英文摘要：

In autonomous driving, 3D occupancy prediction outputs voxel-wise status and semantic labels for more comprehensive understandings of 3D scenes compared with traditional perception tasks, such as 3D object detection and bird's-eye view (BEV) semantic segmentation. Recent researchers have extensively explored various aspects of this task, including view transformation techniques, ground-truth label generation, and elaborate network design, aiming to achieve superior performance. However, the inference speed, crucial for running on an autonomous vehicle, is neglected. To this end, a new method, dubbed FastOcc, is proposed. By carefully analyzing the network effect and latency from four parts, including the input image resolution, image backbone, view transformation, and occupancy prediction head, it is found that the occupancy prediction head holds considerable potential for accelerating the model while keeping its accuracy. Targeted at improving this component, the time-consuming 3D convolution network is replaced with a novel residual-like architecture, where features are mainly digested by a lightweight 2D BEV convolution network and compensated by integrating the 3D voxel features interpolated from the original image features. Experiments on the Occ3D-nuScenes benchmark demonstrate that our FastOcc achieves state-of-the-art results with a fast inference speed.

机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。