PTA-Det:点Transformer关联点云和图像进行三维目标检测
PTA-Det: Point Transformer Associating Point cloud and Image for 3D Object Detection
摘要:
在自动驾驶中,面对车辆周围的复杂环境,基于多模态数据的三维目标检测已成为不可或缺的手段。在多模态检测过程中,激光雷达和相机同时用于捕获和建模。然而,由于激光雷达点云与摄像机图像之间存在固有的差异,使得目标检测的数据融合遇到了一系列问题。大多数多模态检测方法的性能甚至比仅使用激光雷达的方法更差。在这项研究中,我们提出了一种名为PTA-Det的方法来提高多模态检测的性能。在PTA-Det的基础上,提出了一种伪点云生成网络,该网络可以通过伪点来转换图像的纹理和语义特征。然后,通过一个基于变换的点融合转换(PFT)模块,激光雷达点和伪点的图像特征可以在一个统一的基于点的表示下进行深度融合。这些模块的结合可以克服跨模态特征融合的主要障碍,并实现互补和歧视性的建议生成表示。在KITTI数据集上的大量实验表明,PTA-Det实现了具有竞争力的结果,并支持其有效性。
英文摘要:
In autonomous driving, 3D object detection based on multi-modal data has become an indispensable approach when facing complex environments around the vehicle. During multi-modal detection, LiDAR and camera are simultaneously applied for capturing and modeling. However, due to the intrinsic discrepancies between the LiDAR point and camera image, the fusion of the data for object detection encounters a series of problems. Most multi-modal detection methods perform even worse than LiDAR-only methods. In this investigation, we propose a method named PTA-Det to improve the performance of multi-modal detection. Accompanied by PTA-Det, a Pseudo Point Cloud Generation Network is proposed, which can convert image information including texture and semantic features by pseudo points. Thereafter, through a transformer-based Point Fusion Transition (PFT) module, the features of LiDAR points and pseudo points from image can be deeply fused under a unified point-based representation. The combination of these modules can conquer the major obstacle in feature fusion across modalities and realizes a complementary and discriminative representation for proposal generation. Extensive experiments on the KITTI dataset show the PTA-Det achieves a competitive result and support its effectiveness.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。