arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2301.13592cs.CVcs.RO
收藏

先验是强大的:改进Transformer用于具有2D先验的多相机3D检测

Priors are Powerful: Improving a Transformer for Multi-camera 3D Detection with 2D Priors

Argo AI(Argo AI公司);Apple(苹果公司);Nvidia(英伟达)

Di Feng, Francesco Ferroni

摘要:

基于transfomer的方法在学术界和工业界都推进了多相机3D检测的最新发展。在普通的Transformer架构中,查询是随机初始化的,并针对整个数据集进行优化,而不考虑输入帧之间的差异。在这项工作中,我们建议利用预测的图像骨干,这往往是高度优化的2D任务,作为先验的Transformer部分的3D检测网络。该方法通过(1)工作。用2D先验扩充图像特征图,(2).通过沿着2D盒质心的光线投射对查询位置进行采样,以及(3)。用对象级图像特征初始化查询特征。实验结果表明,2D先验不仅有助于模型更快地收敛,而且在平均精度方面大大提高了基线方法高达12%。

英文摘要:

Transfomer-based approaches advance the recent development of multi-camera 3D detection both in academia and industry. In a vanilla transformer architecture, queries are randomly initialised and optimised for the whole dataset, without considering the differences among input frames. In this work, we propose to leverage the predictions from an image backbone, which is often highly optimised for 2D tasks, as priors to the transformer part of a 3D detection network. The method works by (1). augmenting image feature maps with 2D priors, (2). sampling query locations via ray-casting along 2D box centroids, as well as (3). initialising query features with object-level image features. Experimental results shows that 2D priors not only help the model converge faster, but also largely improve the baseline approach by up to 12% in terms of average precision.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑