arXiv 2605.283872026-05-27cs.LGcs.AIcs.NE

CLANE: 基于事件相机在神经形态硬件上的动作持续学习

CLANE: Continual Learning of Actions on Neuromorphic Hardware from Event Cameras

Institute of Informatics, University of Munich (LMU)（慕尼黑大学信息学院）；fortiss GmbH, Neuromorphic Computing（fortiss GmbH 神经形态计算部门）；Technical University of Munich, TUM School of CIT（慕尼黑技术大学 CIT 学院）；Intel Labs, Intel Corporation（英特尔实验室，英特尔公司）；Institute of Computational Life Sciences (ICLS), Zurich University of Applied Sciences (ZHAW)（应用科学大学（ZHAW）计算生命科学研究所）；Technische Universität Ilmenau, Resource-Efficient Artificial Intelligence Group（伊门豪大学资源高效人工智能小组）；Munich Center for Machine Learning (MCML)（慕尼黑机器学习中心）；German Research Centre for Artificial Intelligence (DFKI)（德国人工智能研究中心）

Elvin Hajizada, Michael Neumeier, Edward Paxon Frady, Yulia Sandamirskaya, Axel von Arnim, Bing Li, Eyke Hüllermeier

阅读 PDF

arXiv 页面 ↗HTML 原文 ↗

AI总结：

提出CLANE系统，在Intel Loihi 2神经形态芯片上实现端到端的持续学习，用于事件相机动作识别，通过尖峰CNN和新型Loihi 2模块实现高能效和低延迟。

摘要：

识别并持续学习新的人类动作而不遗忘先前类别，是新兴AR/VR和机器人应用的需求。对于这些应用，设备上的处理和学习对于隐私和低延迟适应至关重要。事件相机通过稀疏、异步的输出解决了视觉传感的效率问题，该输出天然兼容神经形态处理。然而，此前没有系统部署过使用神经形态硬件进行基于事件的持续设备上学习流水线。我们提出了CLANE（基于事件相机在神经形态硬件上的动作持续学习），端到端部署在Intel Loihi 2上。CLANE将用于时空特征提取的脉冲2D CNN与作为片上学习头的CLP-SNN相结合，并通过时间聚合层和定点归一化层（两者均为新型Loihi 2模块）扩展到动作片段。在真实条件下捕获的50类数据集THU E-ACT-50上，CLANE在持续学习任务中达到70.4%的准确率，同时相比顺序CNN+GRU+CLP边缘GPU基线实现了超过100倍的能耗降低和16倍的延迟降低，通过三个评估级别的跨平台等算法基准测试得到验证。

英文摘要：

Recognizing and continuously learning novel human actions without forgetting prior classes is a requirement for emerging AR/VR and robotics applications. For these applications, both on-device processing and learning are essential for privacy and low-latency adaptation. Event cameras address the efficiency of visual sensing with sparse, asynchronous output that is naturally compatible with neuromorphic processing. Yet no prior system has deployed a continual on-device learning pipeline for event-based action recognition using neuromorphic hardware. We present CLANE, Continual Learning of Actions on Neuromorphic Hardware from Event Cameras, deployed end-to-end on Intel Loihi 2. CLANE combines a spiking 2D CNN for spatiotemporal feature extraction with CLP-SNN as its on-chip learning head, extended to action clips via a Temporal Aggregation Layer and a fixed-point Normalization Layer, both novel Loihi 2 modules. On THU E-ACT-50, a 50-class dataset captured under real-world conditions, CLANE achieves 70.4% accuracy in a continual learning task while delivering more than 100x energy reduction and 16x lower latency over a sequential CNN+GRU+CLP edge GPU baseline, validated through iso-algorithm cross-platform benchmarking across three evaluation levels.

机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。