arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2412.02890cs.CV
收藏

EvRT-DETR:面向事件视觉的图像检测器潜在空间适配

EvRT-DETR: Latent Space Adaptation of Image Detectors for Event-based Vision

Brookhaven National Laboratory(布鲁克海文国家实验室)

Dmitrii Torbunov, Yihui Ren, Animesh Ghose, Odera Dim, Yonggang Cui

AI总结:

本文提出I2EvDet框架,通过潜在空间适配将RT-DETR图像检测器转化为事件相机检测模型,在Gen1和Gen4基准上达到最先进性能。

摘要:

事件相机(EBCs)作为一种受生物启发的传统相机替代方案,在功耗效率、时间分辨率和高动态范围方面具有优势。然而,由于事件数据的稀疏性和异步性,为EBCs开发图像分析方法颇具挑战。本研究针对EBC相机的目标检测问题。当前EBC目标检测方法侧重于构建复杂的数据表示,并依赖专用架构。我们提出了I2EvDet(图像到事件检测),一种新颖的适配框架,将主流目标检测与时间事件数据处理相连接。首先,我们证明,在EBC数据的简单图像式表示上训练的实时检测变换器(RT-DETR),一种最先进的自然图像检测器,其性能可与专用EBC方法相媲美。其次,作为框架的一部分,我们开发了一种高效的适配技术,通过最小化架构添加来修改冻结的潜在表示空间,从而将基于图像的检测器转换为基于事件的检测模型。由此产生的EvRT-DETR模型在标准基准数据集Gen1(mAP +2.3)和1Mpx/Gen4(mAP +1.4)上达到了最先进的性能。这些结果展示了一种通过主流架构的原则性适配来实现EBC目标检测的全新方法,提供了一种高效替代方案,并具有应用于其他时间视觉领域的潜力。代码可在 https://github.com/realtime-intelligence/evrt-detr 获取。

英文摘要:

Event-based cameras (EBCs) have emerged as a bio-inspired alternative to traditional cameras, offering advantages in power efficiency, temporal resolution, and high dynamic range. However, the development of image analysis methods for EBCs is challenging due to the sparse and asynchronous nature of the data. This work addresses the problem of object detection for EBC cameras. The current approaches to EBC object detection focus on constructing complex data representations and rely on specialized architectures. We introduce I2EvDet (Image-to-Event Detection), a novel adaptation framework that bridges mainstream object detection with temporal event data processing. First, we demonstrate that a Real-Time DEtection TRansformer, or RT-DETR, a state-of-the-art natural image detector, trained on a simple image-like representation of the EBC data achieves performance comparable to specialized EBC methods. Next, as part of our framework, we develop an efficient adaptation technique that transforms image-based detectors into event-based detection models by modifying their frozen latent representation space through minimal architectural additions. The resulting EvRT-DETR model reaches state-of-the-art performance on the standard benchmark datasets Gen1 (mAP $+2.3$) and 1Mpx/Gen4 (mAP $+1.4$). These results demonstrate a fundamentally new approach to EBC object detection through principled adaptation of mainstream architectures, offering an efficient alternative with potential applications to other temporal visual domains. The code is available at: https://github.com/realtime-intelligence/evrt-detr

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑