arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2102.12319cs.CVcs.RO
收藏

GEM:眩光或昏暗,我仍能看到你--端到端多模态目标检测

GEM: Glare or Gloom, I Can Still See You -- End-to-End Multimodal Object Detection

Delft University of Technology(代尔夫特理工大学);Czech Institute of Informatics, Robotics, and Cybernetics, Czech Technical University in Prague(布拉格捷克理工大学捷克信息学、机器人与控制论研究所)

Osama Mazhar, Robert Babuska, Jens Kober

摘要:

为视觉任务设计的深度神经网络在遇到训练数据未覆盖的环境条件时往往容易失败。当传感器由于故障或其设计限制而无法获取信息时,单模态策略是不够的。已知多传感器配置提供冗余,增加可靠性,并且在实现对不对称传感器故障的鲁棒性方面是至关重要的。针对目标检测中光照条件变化和传感器非对称退化的问题,提出了一种多模态二维目标检测器,并提出了确定性和随机性传感器感知特征融合策略.所提出的融合机制由估计的传感器测量可靠性值/权重驱动。在恶劣的照明条件下进行可靠的目标检测对于自动驾驶车辆和人机交互等应用至关重要。我们还提出了一种新的“r混合”混合深度模式的RGB-D传感器。通过大量的实验,我们表明,所提出的策略优于现有的最先进的方法在FLIR-Thermal数据集上,并获得了有前途的结果在SDGGB-D数据集上。我们还记录了一个新的RGB-Infra室内数据集,即L515-Indoors,并证明了所提出的目标检测方法对于各种照明条件是非常有效的。

英文摘要:

Deep neural networks designed for vision tasks are often prone to failure when they encounter environmental conditions not covered by the training data. Single-modal strategies are insufficient when the sensor fails to acquire information due to malfunction or its design limitations. Multi-sensor configurations are known to provide redundancy, increase reliability, and are crucial in achieving robustness against asymmetric sensor failures. To address the issue of changing lighting conditions and asymmetric sensor degradation in object detection, we develop a multi-modal 2D object detector, and propose deterministic and stochastic sensor-aware feature fusion strategies. The proposed fusion mechanisms are driven by the estimated sensor measurement reliability values/weights. Reliable object detection in harsh lighting conditions is essential for applications such as self-driving vehicles and human-robot interaction. We also propose a new "r-blended" hybrid depth modality for RGB-D sensors. Through extensive experimentation, we show that the proposed strategies outperform the existing state-of-the-art methods on the FLIR-Thermal dataset, and obtain promising results on the SUNRGB-D dataset. We additionally record a new RGB-Infra indoor dataset, namely L515-Indoors, and demonstrate that the proposed object detection methodologies are highly effective for a variety of lighting conditions.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑