arXiv 2411.058022024-10-28cs.NEcs.AIcs.LG

基于相似性的上下文感知脉冲神经网络持续学习

Similarity-based context aware continual learning for spiking neural networks

Institute of Automation, Chinese Academy of Sciences（中国科学院自动化研究所）；University of Chinese Academy of Sciences（中国科学院大学）；Chinese Academy of Sciences（中国科学院）；Center for Long-term Artificial Intelligence（长期人工智能中心）

Bing Han, Feifei Zhao, Yang Li, Qingqun Kong, Xianqi Li, Yi Zeng

阅读 PDF

arXiv 页面 ↗HTML 原文 ↗

AI总结：

提出SCA-SNN，一种基于任务相似性的上下文感知脉冲神经网络持续学习算法，通过选择性重用和扩展神经元，在多个数据集上优于现有SNN和DNN方法，并降低能耗。

摘要：

生物大脑具有根据任务上下文自适应协调相关神经元群体的能力，从而在真实环境中学习持续变化的任务。然而，现有的基于脉冲神经网络的持续学习算法将每个任务同等对待，忽略了不同任务相似性关联对网络学习的引导作用，这限制了知识利用效率。受大脑上下文依赖性可塑性机制的启发，我们提出了一种基于相似性的上下文感知脉冲神经网络（SCA-SNN）持续学习算法，以高效完成任务增量学习和类别增量学习。基于任务间的上下文相似性，SCA-SNN模型能够自适应地重用先前任务中对新任务有益的神经元（相似度越高，重用的神经元越多），并灵活地为新任务扩展新的神经元（相似度越高，扩展的神经元越少）。选择性重用和区分性扩展显著提高了先前知识的利用率，并降低了能量消耗。在CIFAR100、ImageNet广义数据集以及FMNIST-MNIST、SVHN-CIFAR100混合数据集上的大量实验结果表明，我们的SCA-SNN模型在性能上优于基于SNN和基于DNN的持续学习算法。此外，我们的算法具有自适应选择相关任务的相似神经元群体的能力，为增强高效持续学习的生物可解释性提供了一种有前景的方法。

英文摘要：

Biological brains have the capability to adaptively coordinate relevant neuronal populations based on the task context to learn continuously changing tasks in real-world environments. However, existing spiking neural network-based continual learning algorithms treat each task equally, ignoring the guiding role of different task similarity associations for network learning, which limits knowledge utilization efficiency. Inspired by the context-dependent plasticity mechanism of the brain, we propose a Similarity-based Context Aware Spiking Neural Network (SCA-SNN) continual learning algorithm to efficiently accomplish task incremental learning and class incremental learning. Based on contextual similarity across tasks, the SCA-SNN model can adaptively reuse neurons from previous tasks that are beneficial for new tasks (the more similar, the more neurons are reused) and flexibly expand new neurons for the new task (the more similar, the fewer neurons are expanded). Selective reuse and discriminative expansion significantly improve the utilization of previous knowledge and reduce energy consumption. Extensive experimental results on CIFAR100, ImageNet generalized datasets, and FMNIST-MNIST, SVHN-CIFAR100 mixed datasets show that our SCA-SNN model achieves superior performance compared to both SNN-based and DNN-based continual learning algorithms. Additionally, our algorithm has the capability to adaptively select similar groups of neurons for related tasks, offering a promising approach to enhancing the biological interpretability of efficient continual learning.

机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。