arXiv 2003.031432020-03-06cs.LGstat.ML

Triple Memory Networks：一种脑启发的持续学习方法

Triple Memory Networks: a Brain-Inspired Method for Continual Learning

Tsinghua-Peking Center for Life Sciences（清华-北大生命科学联合中心）；IDG/McGovern Institute for Brain Research（IDG/麦戈文脑科学研究院）；School of Life Sciences, Tsinghua University（清华大学生命科学学院）

Liyuan Wang, Bo Lei, Qian Li, Hang Su, Jun Zhu, Yi Zhong

备注：: 12 pages, 3 figures
发表信息：: IEEE Trans Neural Netw Learn Syst. 2021 Sep 16
DOI：: 10.1109/TNNLS.2021.3111019

阅读 PDF

arXiv 页面 ↗HTML 原文 ↗

摘要：

在不干扰先前学习的知识的情况下持续获取新的经验，即持续学习，对于人工神经网络是至关重要的，但受到灾难性遗忘的限制。神经网络在学习新任务时调整其参数，但随后不能很好地执行旧任务。相比之下，大脑有一种强大的能力，可以在不受灾难性干扰的情况下不断学习新的经验。其神经机制可能是由前额叶皮层介导的海马依赖性记忆系统和新皮层依赖性记忆系统的相互作用。具体而言，这两个记忆系统发展专门的机制，以巩固信息的更具体的形式和更普遍的形式，分别，并补充这两种形式的信息的相互作用。受此启发，我们提出了一种新的持续学习方法--三重记忆网络（TMN）。TMN将海马、前额叶皮层和感觉皮层（一个新皮层区域）的相互作用建模为生成对抗网络（GAN）的三重网络架构。输入信息被编码为生成器中的数据分布的特定表示，或者在学习器和分类器中解决任务的一般化知识，并实现适当的大脑启发算法以减轻每个模块中的灾难性遗忘。特别地，生成器将学习到的任务的生成数据重放到分类器和分类器，两者都使用权重合并正则化器来实现，以补充生成过程中丢失的信息。与强基线方法相比，TMN在MNIST，SVHN，CIFAR-10和ImageNet-50上的各种类增量学习基准上实现了新的最先进的性能。

英文摘要：

Continual acquisition of novel experience without interfering previously learned knowledge, i.e. continual learning, is critical for artificial neural networks, but limited by catastrophic forgetting. A neural network adjusts its parameters when learning a new task, but then fails to conduct the old tasks well. By contrast, the brain has a powerful ability to continually learn new experience without catastrophic interference. The underlying neural mechanisms possibly attribute to the interplay of hippocampus-dependent memory system and neocortex-dependent memory system, mediated by prefrontal cortex. Specifically, the two memory systems develop specialized mechanisms to consolidate information as more specific forms and more generalized forms, respectively, and complement the two forms of information in the interplay. Inspired by such brain strategy, we propose a novel approach named triple memory networks (TMNs) for continual learning. TMNs model the interplay of hippocampus, prefrontal cortex and sensory cortex (a neocortex region) as a triple-network architecture of generative adversarial networks (GAN). The input information is encoded as specific representation of the data distributions in a generator, or generalized knowledge of solving tasks in a discriminator and a classifier, with implementing appropriate brain-inspired algorithms to alleviate catastrophic forgetting in each module. Particularly, the generator replays generated data of the learned tasks to the discriminator and the classifier, both of which are implemented with a weight consolidation regularizer to complement the lost information in generation process. TMNs achieve new state-of-the-art performance on a variety of class-incremental learning benchmarks on MNIST, SVHN, CIFAR-10 and ImageNet-50, comparing with strong baseline methods.

机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。

摘要：

英文摘要：

相关论文