arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2303.09919cs.CV
收藏

基于事件的可学习表示目标检测的双记忆聚合网络

Dual Memory Aggregation Network for Event-Based Object Detection with Learnable Representation

School of Artificial Intelligence, Dalian University of Technology(大连理工大学人工智能学院);Advanced Computing and Storage Lab, Huawei Technologies Co. Ltd(华为技术有限公司先进计算与存储实验室)

Dongsheng Wang, Xu Jia, Yang Zhang, Xinyu Zhang, Yaoyuan Wang, Ziyang Zhang, Dong Wang, Huchuan Lu

摘要:

基于事件的相机是生物启发的传感器,以异步方式捕获每个像素的亮度变化。与基于帧的传感器相比,事件相机具有微秒级的延迟和高动态范围,因此在高速运动和光照条件差的情况下显示出巨大的目标检测潜力。由于事件流的稀疏性和非对称性,大多数现有的方法都采用手工制作的方法将事件数据转换为2D网格表示。然而,它们在聚集来自事件流的信息以用于目标检测方面是次优的。在这项工作中,我们建议学习一个事件表示优化基于事件的目标检测。具体而言,事件流被划分为正极性和负极性的x-y-t坐标中的网格,产生一组柱作为3D张量表示。为了充分利用事件流中的信息来检测目标,提出了一种双内存聚合网络(DMANet),利用事件流中的长内存和短内存来聚合有效的信息进行目标检测。长记忆被编码在自适应convLSTM的隐藏状态中,而短记忆通过计算相邻时间间隔的事件柱之间的时空相关性来建模。在最近发布的基于事件的汽车检测数据集上进行的大量实验证明了该方法的有效性。

英文摘要:

Event-based cameras are bio-inspired sensors that capture brightness change of every pixel in an asynchronous manner. Compared with frame-based sensors, event cameras have microsecond-level latency and high dynamic range, hence showing great potential for object detection under high-speed motion and poor illumination conditions. Due to sparsity and asynchronism nature with event streams, most of existing approaches resort to hand-crafted methods to convert event data into 2D grid representation. However, they are sub-optimal in aggregating information from event stream for object detection. In this work, we propose to learn an event representation optimized for event-based object detection. Specifically, event streams are divided into grids in the x-y-t coordinates for both positive and negative polarity, producing a set of pillars as 3D tensor representation. To fully exploit information with event streams to detect objects, a dual-memory aggregation network (DMANet) is proposed to leverage both long and short memory along event streams to aggregate effective information for object detection. Long memory is encoded in the hidden state of adaptive convLSTMs while short memory is modeled by computing spatial-temporal correlation between event pillars at neighboring time intervals. Extensive experiments on the recently released event-based automotive detection dataset demonstrate the effectiveness of the proposed method.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑