ZFusion：一种用于自动驾驶中3D目标感知的相机与4D雷达有效融合器
ZFusion: An Effective Fuser of Camera and 4D Radar for 3D Object Perception in Autonomous Driving
AI总结：
本文提出ZFusion，通过FP-DDCA特征金字塔双重可变形交叉注意力融合稀疏4D雷达与密集视觉特征，并引入Depth-Context-Split视图转换，在VoD数据集上以合理推理速度取得感兴趣区域领先mAP，性能接近LiDAR且优于纯相机方法。
摘要：
可靠的3D目标感知在自动驾驶中至关重要。由于在全天候条件下的传感能力，4D雷达近期备受关注。然而，与LiDAR相比，4D雷达提供的点云稀疏得多。本文提出一种3D目标检测方法，称为ZFusion，融合4D雷达与视觉模态。作为ZFusion的核心，提出的FP-DDCA（Feature Pyramid-Double Deformable Cross Attention）融合器有效互补了（稀疏的）雷达信息与（密集的）视觉信息。具体而言，FP-DDCA融合器借助特征金字塔结构，封装Transformer模块在不同尺度上交互式融合多模态特征，从而提升感知精度。此外，鉴于4D雷达的物理特性，我们采用Depth-Context-Split视角转换模块。考虑到4D雷达成本远低于LiDAR，ZFusion是替代基于LiDAR方法的极具吸引力的方案。在VoD（View-of-Delft）数据集等典型交通场景中，实验表明ZFusion在合理的推理速度下，于感兴趣区域取得了最先进的mAP（mean average precision），同时在整体区域具有与基线方法相竞争的mAP，展现出接近LiDAR的性能并大幅优于纯相机方法。
英文摘要：
Reliable 3D object perception is essential in autonomous driving. Owing to its sensing capabilities in all weather conditions, 4D radar has recently received much attention. However, compared to LiDAR, 4D radar provides much sparser point cloud. In this paper, we propose a 3D object detection method, termed ZFusion, which fuses 4D radar and vision modality. As the core of ZFusion, our proposed FP-DDCA (Feature Pyramid-Double Deformable Cross Attention) fuser complements the (sparse) radar information and (dense) vision information, effectively. Specifically, with a feature-pyramid structure, the FP-DDCA fuser packs Transformer blocks to interactively fuse multi-modal features at different scales, thus enhancing perception accuracy. In addition, we utilize the Depth-Context-Split view transformation module due to the physical properties of 4D radar. Considering that 4D radar has a much lower cost than LiDAR, ZFusion is an attractive alternative to LiDAR-based methods. In typical traffic scenarios like the VoD (View-of-Delft) dataset, experiments show that with reasonable inference speed, ZFusion achieved the state-of-the-art mAP (mean average precision) in the region of interest, while having competitive mAP in the entire area compared to the baseline methods, which demonstrates performance close to LiDAR and greatly outperforms those camera-only methods.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
