arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2211.14710cs.CV
收藏

3DPPE:用于多相机3D目标检测Transformer的3D点位置编码

3DPPE: 3D Point Positional Encoding for Multi-Camera 3D Object Detection Transformers

Houmo AI(厚墨人工智能);University of Sydney(悉尼大学);ETH Zürich(苏黎世联邦理工学院);University of Adelaide(阿德莱德大学)

Changyong Shu, JIajun Deng, Fisher Yu, Yifan Liu

AI总结:

提出3DPPE方法,通过预测深度近似3D点位置并引入3D检测Transformer解码器,在nuScenes数据集上显著提升多相机3D目标检测性能。

摘要:

基于Transformer的方法席卷了图像2D和3D检测的基准。由于注意力机制之前的分词化会丢失空间信息,位置编码对这些方法至关重要。近期研究发现,基于3D视线采样的编码能显著提升多相机3D目标检测质量。我们假设3D点位置能比视线提供更多信息。因此,我们将3D点位置编码(3DPPE)引入3D检测Transformer解码器。尽管单目3D目标检测推理时无法获取3D测量值,3DPPE仍利用预测深度来近似真实点位置。我们的混合深度模块结合直接深度和分类深度,估计每个像素的精细深度。尽管存在近似,3DPPE在竞争激烈的nuScenes数据集上达到46.0 mAP和51.4 NDS,显著优于基于视线采样的编码。代码发布于https://github.com/drilistbox/3DPPE。

英文摘要:

Transformer-based methods have swept the benchmarks on 2D and 3D detection on images. Because tokenization before the attention mechanism drops the spatial information, positional encoding becomes critical for those methods. Recent works found that encodings based on samples of the 3D viewing rays can significantly improve the quality of multi-camera 3D object detection. We hypothesize that 3D point locations can provide more information than rays. Therefore, we introduce 3D point positional encoding, 3DPPE, to the 3D detection Transformer decoder. Although 3D measurements are not available at the inference time of monocular 3D object detection, 3DPPE uses predicted depth to approximate the real point positions. Our hybriddepth module combines direct and categorical depth to estimate the refined depth of each pixel. Despite the approximation, 3DPPE achieves 46.0 mAP and 51.4 NDS on the competitive nuScenes dataset, significantly outperforming encodings based on ray samples. We make the codes available at https://github.com/drilistbox/3DPPE.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑