从事件中检测每个物体
Detecting Every Object from Events
AI总结:
针对普通相机在未知类别目标检测中的高延迟和低动态范围问题,提出基于事件相机的 DEOE,结合时空一致性、软正样本与解耦 objectness 头,实现高速开放世界类别无关检测。
摘要:
目标检测在自动驾驶中至关重要,而定位未知类别的物体更加实用且具有挑战性:这一任务被称为类别无关目标检测(Class-Agnostic Object Detection,CAOD)。现有的 CAOD 研究主要依赖普通相机,但这些基于帧的传感器通常具有较高延迟和有限的动态范围,在真实场景中会带来安全风险。在本研究中,我们转向由事件相机提供支持的一种新模态,该模态以亚毫秒级延迟和高动态范围为特征,用于实现鲁棒的 CAOD。我们提出了 Detecting Every Object in Events(DEOE),这是一种专为在事件视觉中实现高速、类别无关的开放世界目标检测而定制的方法。该方法构建在快速的事件主干网络——循环视觉 Transformer——之上,联合考虑空间和时间一致性以识别潜在物体。所发现的潜在物体被作为软正样本吸收,以避免其被当作背景抑制。此外,我们引入了解耦的 objectness 头,用于分离前景-背景分类与新物体发现任务,从而增强模型定位新物体的泛化能力,同时保持强大的背景过滤能力。大量实验证实,与三种将最先进的基于事件的目标检测器与基于 RGB 的 CAOD 进展相结合的强基线方法相比,我们提出的 DEOE 具有优越性。我们的代码可在 https://github.com/Hatins/DEOE 获取。
英文摘要:
Object detection is critical in autonomous driving, and it is more practical yet challenging to localize objects of unknown categories: an endeavour known as Class-Agnostic Object Detection (CAOD). Existing studies on CAOD predominantly rely on ordinary cameras, but these frame-based sensors usually have high latency and limited dynamic range, leading to safety risks in real-world scenarios. In this study, we turn to a new modality enabled by the so-called event camera, featured by its sub-millisecond latency and high dynamic range, for robust CAOD. We propose Detecting Every Object in Events (DEOE), an approach tailored for achieving high-speed, class-agnostic open-world object detection in event-based vision. Built upon the fast event-based backbone: recurrent vision transformer, we jointly consider the spatial and temporal consistencies to identify potential objects. The discovered potential objects are assimilated as soft positive samples to avoid being suppressed as background. Moreover, we introduce a disentangled objectness head to separate the foreground-background classification and novel object discovery tasks, enhancing the model's generalization in localizing novel objects while maintaining a strong ability to filter out the background. Extensive experiments confirm the superiority of our proposed DEOE in comparison with three strong baseline methods that integrate the state-of-the-art event-based object detector with advancements in RGB-based CAOD. Our code is available at https://github.com/Hatins/DEOE.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。