arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2304.11697cs.CVeess.IV
收藏

面向多模态目标检测的带不确定性的信息性数据选择

Informative Data Selection with Uncertainty for Multi-modal Object Detection

Beihang University(北京航空航天大学);Tsinghua University(清华大学);China University of Mining & Technology, Beijing(中国矿业大学(北京));Beijing University of Chemical Technology(北京化工大学)

Xinyu Zhang, Zhiwei Li, Zhenhong Zou, Xin Gao, Yijin Xiong, Dafeng Jin, Jun Li, Huaping Liu

AI总结:

本文提出不确定性感知的多流水线松耦合融合模型,通过将不确定性作为数据信息的逆嵌入边界框生成,在 KITTI 噪声数据上实现稳健的点云-图像目标检测。

摘要:

噪声始终是目标检测中不可忽视的问题,它会在模型推理过程中造成混淆,从而降低数据的信息性。由于观测到的模式发生偏移,噪声可能导致不准确的识别,这要求模型具备稳健的泛化能力。为实现通用视觉模型,我们需要开发能够从多模态数据中自适应选择有效信息的深度学习模型。这主要基于两个原因:多模态学习能够突破单模态数据的固有缺陷,而自适应信息选择可以减少多模态数据中的混乱。为解决这一问题,我们提出了一种通用的不确定性感知多模态融合模型。该模型采用多流水线松耦合架构,以结合来自点云和图像的特征与结果。为量化多模态信息中的相关性,我们将不确定性建模为数据信息的逆,并将其嵌入边界框生成过程。通过这种方式,我们的模型减少了融合中的随机性,并生成可靠的输出。此外,我们在 KITTI 2D 目标检测数据集及其派生的脏数据上开展了完整研究。实验证明,我们的融合模型能够抵抗 Gaussian、motion blur 和 frost 等严重噪声干扰,且仅有轻微性能下降。实验结果证明了我们自适应融合的优势。我们对多模态融合稳健性的分析将为未来研究提供进一步洞见。

英文摘要:

Noise has always been nonnegligible trouble in object detection by creating confusion in model reasoning, thereby reducing the informativeness of the data. It can lead to inaccurate recognition due to the shift in the observed pattern, that requires a robust generalization of the models. To implement a general vision model, we need to develop deep learning models that can adaptively select valid information from multi-modal data. This is mainly based on two reasons. Multi-modal learning can break through the inherent defects of single-modal data, and adaptive information selection can reduce chaos in multi-modal data. To tackle this problem, we propose a universal uncertainty-aware multi-modal fusion model. It adopts a multi-pipeline loosely coupled architecture to combine the features and results from point clouds and images. To quantify the correlation in multi-modal information, we model the uncertainty, as the inverse of data information, in different modalities and embed it in the bounding box generation. In this way, our model reduces the randomness in fusion and generates reliable output. Moreover, we conducted a completed investigation on the KITTI 2D object detection dataset and its derived dirty data. Our fusion model is proven to resist severe noise interference like Gaussian, motion blur, and frost, with only slight degradation. The experiment results demonstrate the benefits of our adaptive fusion. Our analysis on the robustness of multi-modal fusion will provide further insights for future research.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑