LiCROcc：使用LiDAR和相机教导雷达进行准确的语义占用预测
LiCROcc: Teach Radar for Accurate Semantic Occupancy Prediction using LiDAR and Camera
AI总结：
本文提出LiCROcc框架，通过跨模态蒸馏将LiDAR和相机融合特征的知识传递给雷达模型，在nuScenes-Occupancy数据集上显著提升了语义场景补全性能。
摘要：
语义场景补全（SSC）在自动驾驶感知中至关重要，经常面临天气和光照变化的复杂性。长期策略涉及融合多模态信息以增强系统的鲁棒性。雷达越来越多地用于3D目标检测，并逐渐在自动驾驶应用中取代LiDAR，提供了一种鲁棒的传感替代方案。在本文中，我们关注3D雷达在语义场景补全中的潜力，开创跨模态细化技术以提高对天气和光照变化的鲁棒性，并提升SSC性能。在模型架构方面，我们提出了一种在BEV上的三阶段紧密融合方法，以实现点云和图像的融合框架。基于此基础，我们设计了三个跨模态蒸馏模块——CMRD、BRD和PDD。我们的方法通过将LiDAR和相机融合特征的丰富语义和结构信息蒸馏到它们中，提升了纯雷达（R-LiCROcc）和雷达-相机（RC-LiCROcc）设置下的性能。最后，我们的LC-Fusion（教师模型）、R-LiCROcc和RC-LiCROcc在nuScenes-Occupancy数据集上取得了最佳性能，mIOU分别超过基线22.9%、44.1%和15.5%。项目页面可在https://hr-zju.github.io/LiCROcc/获取。
英文摘要：
Semantic Scene Completion (SSC) is pivotal in autonomous driving perception, frequently confronted with the complexities of weather and illumination changes. The long-term strategy involves fusing multi-modal information to bolster the system's robustness. Radar, increasingly utilized for 3D target detection, is gradually replacing LiDAR in autonomous driving applications, offering a robust sensing alternative. In this paper, we focus on the potential of 3D radar in semantic scene completion, pioneering cross-modal refinement techniques for improved robustness against weather and illumination changes, and enhancing SSC performance.Regarding model architecture, we propose a three-stage tight fusion approach on BEV to realize a fusion framework for point clouds and images. Based on this foundation, we designed three cross-modal distillation modules-CMRD, BRD, and PDD. Our approach enhances the performance in both radar-only (R-LiCROcc) and radar-camera (RC-LiCROcc) settings by distilling to them the rich semantic and structural information of the fused features of LiDAR and camera. Finally, our LC-Fusion (teacher model), R-LiCROcc and RC-LiCROcc achieve the best performance on the nuScenes-Occupancy dataset, with mIOU exceeding the baseline by 22.9%, 44.1%, and 15.5%, respectively. The project page is available at https://hr-zju.github.io/LiCROcc/.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
