互补学习系统的神经网络模型:用于持续学习的模式分离与模式补全
A Neural Network Model of Complementary Learning Systems: Pattern Separation and Completion for Continual Learning
AI总结:
针对神经网络灾难性遗忘问题,结合变分自编码器与现代Hopfield网络构建互补学习系统模型,实现模式补全与分离,在Split-MNIST上准确率接近90%并大幅减少遗忘。
摘要:
在不遗忘先验知识的情况下学习新信息是人类智能的核心。相比之下,神经网络模型会遭受灾难性遗忘:在获取新信息时,先前学习任务上的性能会显著下降。互补学习系统(CLS)理论为人类的这种能力提供了解释,提出大脑具有不同的系统用于模式分离(编码不同的记忆)和模式补全(从部分线索中检索完整记忆)。为了捕捉这些互补功能,我们利用变分自编码器(VAEs)的表征泛化能力和现代Hopfield网络(MHNs)的鲁棒记忆存储特性,将它们结合成一个神经上合理的持续学习模型。我们在流行的持续学习基准Split-MNIST任务上评估了该模型,并实现了接近最先进的准确率(~90%),大幅减少了遗忘。表征分析在经验上证实了这种功能分离:VAE支持模式补全,而MHN驱动模式分离。通过在可扩展架构中捕捉模式分离和补全,我们的工作为在生物和人工系统中建模记忆巩固、泛化和持续学习提供了一个功能模板。
英文摘要:
Learning new information without forgetting prior knowledge is central to human intelligence. In contrast, neural network models suffer from catastrophic forgetting: a significant degradation in performance on previously learned tasks when acquiring new information. The Complementary Learning Systems (CLS) theory offers an explanation for this human ability, proposing that the brain has distinct systems for pattern separation (encoding distinct memories) and pattern completion (retrieving complete memories from partial cues). To capture these complementary functions, we leverage the representational generalization capabilities of variational autoencoders (VAEs) and the robust memory storage properties of Modern Hopfield networks (MHNs), combining them into a neurally plausible continual learning model. We evaluate this model on the Split-MNIST task, a popular continual learning benchmark, and achieve close to state-of-the-art accuracy (~90%), substantially reducing forgetting. Representational analyses empirically confirm the functional dissociation: the VAE underwrites pattern completion, while the MHN drives pattern separation. By capturing pattern separation and completion in scalable architectures, our work provides a functional template for modeling memory consolidation, generalization, and continual learning in both biological and artificial systems.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。