arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2303.01686cs.CV
收藏

多视角三维目标检测的领域泛化方法

Towards Domain Generalization for Multi-view 3D Object Detection in Bird-Eye-View

University of Adelaide(阿德莱德大学);University of Science and Technology of China(中国科学技术大学);Huawei Noah’s Ark Lab(华为诺亚方舟实验室)

Shuo Wang, Xinhai Zhao, Hai-Ming Xu, Zehui Chen, Dameng Yu, Jiahao Chang, Zhen Yang, Feng Zhao

摘要:

多视角三维目标检测(MV 3D-Det)以其低成本、高效率的特点受到广泛关注。虽然已经不断提出了用于仅摄像机的3D目标检测的新算法,但是当输入图像的域与训练图像的域不同时,它们中的大多数可能会有性能急剧下降的风险。在本文中,我们首先分析了MV 3D-Det任务的域间隙的原因。基于协变量偏移假设,我们发现这种差距主要归因于BEV的特征分布,而BEV的特征分布由深度估计和二维图像特征表示的质量共同决定。为了获得一个强大的深度预测,我们建议解耦的深度估计的内部参数的相机(即焦距)通过转换的预测度量深度的尺度不变的深度和执行动态的透视增强,以增加外部参数(即相机姿态)的多样性,通过利用单应性。此外,我们修改焦距值以创建多个伪域,并构建对抗性训练损失,以鼓励特征表示更加领域不可知。在没有花里胡哨的情况下,我们的方法,即DG-BEV,成功地消除了看不见的目标域上的性能下降,而不会损害源域的准确性。在各种公共数据集(包括Waymo,nuScenes和Lyft)上进行的大量实验证明了我们方法的通用性和有效性。据我们所知,这是第一个系统的研究,探索领域泛化方法的MV 3D-Det。

英文摘要:

Multi-view 3D object detection (MV3D-Det) in Bird-Eye-View (BEV) has drawn extensive attention due to its low cost and high efficiency. Although new algorithms for camera-only 3D object detection have been continuously proposed, most of them may risk drastic performance degradation when the domain of input images differs from that of training. In this paper, we first analyze the causes of the domain gap for the MV3D-Det task. Based on the covariate shift assumption, we find that the gap mainly attributes to the feature distribution of BEV, which is determined by the quality of both depth estimation and 2D image's feature representation. To acquire a robust depth prediction, we propose to decouple the depth estimation from the intrinsic parameters of the camera (i.e. the focal length) through converting the prediction of metric depth to that of scale-invariant depth and perform dynamic perspective augmentation to increase the diversity of the extrinsic parameters (i.e. the camera poses) by utilizing homography. Moreover, we modify the focal length values to create multiple pseudo-domains and construct an adversarial training loss to encourage the feature representation to be more domain-agnostic. Without bells and whistles, our approach, namely DG-BEV, successfully alleviates the performance drop on the unseen target domain without impairing the accuracy of the source domain. Extensive experiments on various public datasets, including Waymo, nuScenes, and Lyft, demonstrate the generalization and effectiveness of our approach. To the best of our knowledge, this is the first systematic study to explore a domain generalization method for MV3D-Det.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑