arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2310.02642cs.CVcs.AI
收藏

GET:面向事件相机视觉的 Group Event Transformer

GET: Group Event Transformer for Event-Based Vision

University of Science Technology of China(中国科学技术大学)

Yansong Peng, Yueyi Zhang, Zhiwei Xiong, Xiaoyan Sun, Feng Wu

AI总结:

本文提出 Group Event Transformer(GET),通过 Group Token、Event Dual Self-Attention 和聚合模块解耦并融合空间与时间-极性信息,在多个事件分类和检测数据集上取得最优性能。

摘要:

事件相机是一种新型神经形态传感器,正受到越来越多的关注。现有的基于事件的骨干网络主要依赖面向图像的设计,在由事件转换得到的图像中提取空间信息,却忽视了时间和极性等重要事件属性。为解决这一问题,我们提出了一种新颖的、面向基于事件视觉的基于分组的视觉 Transformer 骨干网络,称为 Group Event Transformer(GET),它在整个特征提取过程中将时间-极性信息与空间信息解耦。具体而言,我们首先为 GET 提出了一种名为 Group Token 的新事件表示,它根据异步事件的时间戳和极性对其进行分组。随后,GET 采用 Event Dual Self-Attention 模块和 Group Token Aggregation 模块,以促进空间域与时间-极性域中有效的特征通信和整合。之后,GET 可以通过连接各种任务头来集成到不同的下游任务中。我们在四个基于事件的分类数据集(Cifar10-DVS、N-MNIST、N-CARS 和 DVS128Gesture)以及两个基于事件的目标检测数据集(1Mpx 和 Gen1)上评估了所提方法,结果表明 GET 优于其他最先进的方法。代码可在 https://github.com/Peterande/GET-Group-Event-Transformer 获取。

英文摘要:

Event cameras are a type of novel neuromorphic sen-sor that has been gaining increasing attention. Existing event-based backbones mainly rely on image-based designs to extract spatial information within the image transformed from events, overlooking important event properties like time and polarity. To address this issue, we propose a novel Group-based vision Transformer backbone for Event-based vision, called Group Event Transformer (GET), which de-couples temporal-polarity information from spatial infor-mation throughout the feature extraction process. Specifi-cally, we first propose a new event representation for GET, named Group Token, which groups asynchronous events based on their timestamps and polarities. Then, GET ap-plies the Event Dual Self-Attention block, and Group Token Aggregation module to facilitate effective feature commu-nication and integration in both the spatial and temporal-polarity domains. After that, GET can be integrated with different downstream tasks by connecting it with vari-ous heads. We evaluate our method on four event-based classification datasets (Cifar10-DVS, N-MNIST, N-CARS, and DVS128Gesture) and two event-based object detection datasets (1Mpx and Gen1), and the results demonstrate that GET outperforms other state-of-the-art methods. The code is available at https://github.com/Peterande/GET-Group-Event-Transformer.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑