arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2211.08055cs.CV
收藏

PAI3D:用于3D目标检测的绘制自适应实例先验

PAI3D: Painting Adaptive Instance-Prior for 3D Object Detection

JD Logistics(京东物流)

Hao Liu, Zhuoran Xu, Dan Wang, Baofeng Zhang, Guan Wang, Bo Dong, Xin Wen, Xinyu Xu

摘要:

3D目标检测是自动驾驶中的一项关键任务。近年来,基于多模态融合的三维目标检测方法结合了激光雷达和摄像机的优势互补,相对于单模态方法有了很大的性能提升。然而,到目前为止,还没有方法尝试利用实例级上下文图像语义来指导3D目标检测。本文提出了一种简单有效的Painting Adaptive Instance-prior for 3D目标检测(PAI 3D)方法,将实例级图像语义与点云特征灵活融合。PAI 3D是一个多模态顺序实例级融合框架。该方法首先从图像中提取实例级语义信息,然后利用提取的信息(包括对象类别标签、点到对象隶属度和对象位置)对后续三维检测网络中的每个LiDAR点进行增强,以指导和提高检测性能。PAI 3D在nuScenes数据集上的表现优于最先进的技术,在测试分割上达到mAP 71.4和NDS 74.2。我们的综合实验表明,实例级图像语义对性能增益的贡献最大,PAI 3D与任何高质量的实例分割模型和任何现代点云3D编码器都能很好地配合使用,使其成为部署在自动驾驶汽车上的强有力的候选者。

英文摘要:

3D object detection is a critical task in autonomous driving. Recently multi-modal fusion-based 3D object detection methods, which combine the complementary advantages of LiDAR and camera, have shown great performance improvements over mono-modal methods. However, so far, no methods have attempted to utilize the instance-level contextual image semantics to guide the 3D object detection. In this paper, we propose a simple and effective Painting Adaptive Instance-prior for 3D object detection (PAI3D) to fuse instance-level image semantics flexibly with point cloud features. PAI3D is a multi-modal sequential instance-level fusion framework. It first extracts instance-level semantic information from images, the extracted information, including objects categorical label, point-to-object membership and object position, are then used to augment each LiDAR point in the subsequent 3D detection network to guide and improve detection performance. PAI3D outperforms the state-of-the-art with a large margin on the nuScenes dataset, achieving 71.4 in mAP and 74.2 in NDS on the test split. Our comprehensive experiments show that instance-level image semantics contribute the most to the performance gain, and PAI3D works well with any good-quality instance segmentation models and any modern point cloud 3D encoders, making it a strong candidate for deployment on autonomous vehicles.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑