基于多帧表示预训练的事件驱动动作识别
Event-based Action Recognition by Pre-Training on Multi-Frame Representations
AI总结：
本研究针对事件驱动动作识别中长序列判别能力不足的问题，提出多帧表示方法，采用掩码自编码框架预训练视觉Transformer，在THU-EACT-50 CHL和DailyDVS-200数据集上取得当前最优性能。
摘要：
动作识别是机器人技术、患者监护或监控等领域的关键任务。近年来，事件驱动动作识别因事件相机能捕捉快速运动且相比传统视觉相机隐私性更好而受到广泛关注。尽管基于帧的事件表示在短事件序列中表现良好，但在更长、更复杂的事件序列中其判别能力会急剧下降。本研究提出一种多帧表示，相比单帧表示大幅提升识别性能。我们采用掩码自编码框架预训练视觉Transformer，使用感知稀疏性的掩码算法和令牌选择机制提升下游性能并控制计算复杂度；还为解码器引入多个掩码令牌，搭配位置感知注意力机制进一步提升性能。所提方法在具有挑战性的长时长、复杂事件序列上表现出色，在THU-EACT-50 CHL和DailyDVS-200数据集上取得了当前最优性能。
英文摘要：
Action recognition is a crucial task in domains like robotics, patient monitoring, or surveillance. Recently, event-based action recognition has attracted much attention due to the event camera's ability to capture fast motions and its improved privacy over conventional vision cameras. While frame-based event representations work well for short event sequences, their discriminative ability declines sharply with longer and more complex event sequences. In this work, we introduce a multi-frame representation that greatly improves recognition performance over its single-frame counterpart. We adopt a masked auto-encoding framework to pre-train a Vision Transformer using a sparsity-aware masking algorithm and token selection mechanism to improve downstream performance and control computational complexity. We also introduce multiple mask tokens for the decoder with a position-aware attention mechanism, leading to further improved performance. Our proposed method shows impressive performance for challenging long-duration and complex event sequences, yielding state-of-the-art performance on THU-EACT-50 CHL and DailyDVS-200.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
