抑制神经网络遗忘的局部学习规则
Local learning rules to attenuate forgetting in neural networks
摘要：
当不同的、重叠的记忆模式顺序地存储在同一个网络中时，赫布突触可塑性不可避免地会导致干扰和遗忘。最近的人工神经网络的工作表明，信息几何方法可以用来保护重要的权重，以减缓遗忘。然而，这种策略在生物学上是不可信的，因为它需要了解以前学习的模式的历史。在这项工作中，我们表明，一个纯粹的本地权重合并机制，估计能量景观曲率从本地可用的统计数据的基础上，防止模式干扰。探索在稀疏编码限制下的能量曲率的局部计算，我们证明了曲率感知的学习规则减少了Hopfield网络中的遗忘。我们进一步表明，该方法连接信息几何的全局学习规则的基础上的Fisher信息的局部尖峰依赖的规则，生物神经网络。我们推测，如果与其他学习过程相结合，它可以提供一个构建块的内容感知学习策略，只使用生物神经网络中的可计算量，以减弱模式干扰和灾难性遗忘。此外，这项工作阐明了如何在局部模型统计中暴露学习问题中的全局信息几何结构，从而在网络中单个单元的统计数据与集体学习空间的全局结构之间建立更深层次的理论联系。
英文摘要：
Hebbian synaptic plasticity inevitably leads to interference and forgetting when different, overlapping memory patterns are sequentially stored in the same network. Recent work on artificial neural networks shows that an information-geometric approach can be used to protect important weights to slow down forgetting. This strategy however is biologically implausible as it requires knowledge of the history of previously learned patterns. In this work, we show that a purely local weight consolidation mechanism, based on estimating energy landscape curvatures from locally available statistics, prevents pattern interference. Exploring a local calculation of energy curvature in the sparse-coding limit, we demonstrate that curvature-aware learning rules reduce forgetting in the Hopfield network. We further show that this method connects information-geometric global learning rules based on the Fisher information to local spike-dependent rules accessible to biological neural networks. We conjecture that, if combined with other learning procedures, it could provide a building-block for content-aware learning strategies that use only quantities computable in biological neural networks to attenuate pattern interference and catastrophic forgetting. Additionally, this work clarifies how global information-geometric structure in a learning problem can be exposed in local model statistics, building a deeper theoretical connection between the statistics of single units in a network, and the global structure of the collective learning space.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
