基于赫布学习的正交投影用于脉冲神经网络的持续学习
Hebbian Learning based Orthogonal Projection for Continual Learning of Spiking Neural Networks
AI总结：
针对脉冲神经网络的灾难性遗忘问题，提出基于侧向连接和赫布学习的正交投影方法，通过提取主子空间保护旧知识，在多种监督训练下实现接近零遗忘并优于现有方法。
摘要：
带有脉冲神经网络的神经形态计算对于节能的人工智能（AI）应用具有广阔前景。然而，与人类在一生中不断学习不同任务不同，神经网络模型会遭受灾难性遗忘。神经元操作如何解决这一问题，是AI和神经科学领域的一个重要问题。许多先前的研究从观察到的神经科学现象中汲取灵感，提出了情景回放或突触可塑性，但它们并不能保证显式地保留神经元群体的知识。其他工作则专注于具有更多数学基础的机器学习方法，例如高维空间中的正交投影，但在神经形态计算中缺乏对应的神经机制。在本研究中，我们开发了一种基于侧向连接和赫布学习的神经元操作新方法，该方法通过将神经元的活动迹投影到正交子空间来保护知识，从而使突触权重更新不会干扰旧任务。我们证明了在循环侧向连接上的赫布和反赫布学习可以有效地提取神经活动的主子空间并实现正交投影。这为神经回路和赫布学习如何帮助持续学习，以及正交投影的概念如何在神经元系统中实现提供了新见解。我们的方法也很灵活，能够利用基于突触前活动/迹的任意训练方法。实验表明，我们的方法在采用不同误差传播方法的各种监督训练下，能够持续解决脉冲神经网络的遗忘问题，实现接近零遗忘，并在各种设置下优于先前的方法。我们的方法为构建持续神经形态计算系统铺平了坚实的道路。
英文摘要：
Neuromorphic computing with spiking neural networks is promising for energy-efficient artificial intelligence (AI) applications. However, different from humans who continually learn different tasks in a lifetime, neural network models suffer from catastrophic forgetting. How could neuronal operations solve this problem is an important question for AI and neuroscience. Many previous studies draw inspiration from observed neuroscience phenomena and propose episodic replay or synaptic metaplasticity, but they are not guaranteed to explicitly preserve knowledge for neuron populations. Other works focus on machine learning methods with more mathematical grounding, e.g., orthogonal projection on high dimensional spaces, but there is no neural correspondence for neuromorphic computing. In this work, we develop a new method with neuronal operations based on lateral connections and Hebbian learning, which can protect knowledge by projecting activity traces of neurons into an orthogonal subspace so that synaptic weight update will not interfere with old tasks. We show that Hebbian and anti-Hebbian learning on recurrent lateral connections can effectively extract the principal subspace of neural activities and enable orthogonal projection. This provides new insights into how neural circuits and Hebbian learning can help continual learning, and also how the concept of orthogonal projection can be realized in neuronal systems. Our method is also flexible to utilize arbitrary training methods based on presynaptic activities/traces. Experiments show that our method consistently solves forgetting for spiking neural networks with nearly zero forgetting under various supervised training methods with different error propagation approaches, and outperforms previous approaches under various settings. Our method can pave a solid path for building continual neuromorphic computing systems.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
