基于事件数据与深度数据传感器融合的脉冲卷积网络高效手势识别
Efficient Gesture Recognition on Spiking Convolutional Networks Through Sensor Fusion of Event-Based and Depth Data
AI总结：
针对传统手势识别反应不足的问题，提出融合事件与深度数据的脉冲卷积网络，用LAVA框架仿真，结合新录制的同步双模态数据集验证，证实时间编码与模态融合可提升性能与泛化能力。
摘要：
随着智能系统在日常生活中愈发重要，人们需要新的交互方式。经典用户界面给身体障碍人士带来困扰，且部分场景下不够实用或便捷。手势识别是一种替代方案，但使用传统相机时往往反应不够灵敏。本研究提出一种脉冲卷积神经网络，通过处理事件数据与深度数据实现手势识别。该网络采用开源神经形态计算框架LAVA进行仿真，以在嵌入式系统上完成离线训练与评估。评估阶段使用了三个开源数据集，由于这些数据集不具备所需的双模态特性，研究人员录制了一个包含同步事件数据与深度数据的新数据集。结果表明，对深度信息进行时间编码以及模态融合（即便针对编码方式不同的数据）均有助于提升网络性能与泛化能力。
英文摘要：
As intelligent systems become increasingly important in our daily lives, new ways of interaction are needed. Classical user interfaces pose issues for the physically impaired and are partially not practical or convenient. Gesture recognition is an alternative, but often not reactive enough when conventional cameras are used. This work proposes a Spiking Convolutional Neural Network, processing event- and depth data for gesture recognition. The network is simulated using the open-source neuromorphic computing framework LAVA for offline training and evaluation on an embedded system. For the evaluation three open source data sets are used. Since these do not represent the applied bi-modality, a new data set with synchronized event- and depth data was recorded. The results show the viability of temporal encoding on depth information and modality fusion, even on differently encoded data, to be beneficial to network performance and generalization capabilities.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
