arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2402.02026cs.CVcs.AI
收藏

面向自动驾驶中边角案例检测的多模态增强物体性学习器

Multimodal-Enhanced Objectness Learner for Corner Case Detection in Autonomous Driving

School of Computer Science and Engineering, Southeast University(东南大学计算机科学与工程学院)

Lixing Xiao, Ruixiao Shi, Xiaoyang Tang, Yi Zhou

AI总结:

本文提出MENOL,通过多模态半监督学习缩小已知与未知类差异,在仅5100张标注图像下显著提升自动驾驶边角案例检测召回率,分别超越基线ORE 71.3%和60.6%。

摘要:

以往的目标检测工作在封闭集场景中已取得高精度,但在开放世界场景中的表现并不令人满意。其中一个具有挑战性的开放世界问题是自动驾驶中的边角案例检测。现有检测器在这些案例上表现挣扎,严重依赖视觉外观,且泛化能力较差。本文提出一种通过缩小已知类与未知类之间差异的解决方案,并引入一种多模态增强的物体性概念学习器。我们的半监督学习框架利用视觉中心模态和图像-文本模态,将物体性知识传授给学生模型,从而实现类别感知检测。我们的方法,即面向边角案例检测的多模态增强物体性学习器(MENOL),以更低的训练成本显著提升了新类别的召回率。仅在CODA-val数据集上使用5100张带标签训练图像,MENOL就取得了76.6%的mAR-corner和79.8%的mAR-agnostic,分别比基线ORE高出71.3%和60.6%。代码将发布在https://github.com/tryhiseyyysum/MENOL。

英文摘要:

Previous works on object detection have achieved high accuracy in closed-set scenarios, but their performance in open-world scenarios is not satisfactory. One of the challenging open-world problems is corner case detection in autonomous driving. Existing detectors struggle with these cases, relying heavily on visual appearance and exhibiting poor generalization ability. In this paper, we propose a solution by reducing the discrepancy between known and unknown classes and introduce a multimodal-enhanced objectness notion learner. Leveraging both vision-centric and image-text modalities, our semi-supervised learning framework imparts objectness knowledge to the student model, enabling class-aware detection. Our approach, Multimodal-Enhanced Objectness Learner (MENOL) for Corner Case Detection, significantly improves recall for novel classes with lower training costs. By achieving a 76.6% mAR-corner and 79.8% mAR-agnostic on the CODA-val dataset with just 5100 labeled training images, MENOL outperforms the baseline ORE by 71.3% and 60.6%, respectively. The code will be available at https://github.com/tryhiseyyysum/MENOL.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑