对终身学习的两种思考：探索神经模型中的半球冗余与特化
In Two Minds about Lifelong Learning: Exploring Hemispheric Redundancy and Specialisation in Neural Models
浏览论文内容
中文总结 AI 辅助
本文针对机器学习的持续学习挑战，提出4MAS架构，结合经验重放、REM睡眠与双侧性，利用不对称半球的长短时记忆机制，在三个分割数据集上取得有竞争力的准确率。
中文摘要 AI 辅助
持久智能系统需要具备持续学习的能力，但当前机器学习方法在这方面与生物学习系统相比面临重大挑战。机器学习算法通常需要在保留先前学习的信息与适应新的或变化的数据模式之间进行权衡。当缺乏持续学习能力时，算法必须使用整个数据集进行重新训练，由于存储限制、财务或计算成本或隐私限制导致原始训练数据不可用，这种方法变得不切实际。然而，生物动物可以持续学习，而不会遭受灾难性遗忘。本文尝试通过建模已知与记忆巩固相关的神经组件和状态，构建动物学习和保存知识的高级框架。我们聚焦三个概念：经验重放、REM睡眠和双侧性。我们提出4MAS（4模块清醒/睡眠），一种新型宏观架构，展示机器学习模型如何从不对称半球中受益，每个半球具有各自的长短期记忆机制，以及增量学习任务之间的睡眠期如何有益于记忆巩固。最后，我们展示结果，表明我们的架构在Split-MNIST、Split-Fashion-MNIST和Split-CIFAR-100数据集上取得了有竞争力的结果，准确率分别为98.3%、84.9%和29.29%。
英文摘要
Persistent intelligent systems require the ability to learn continually, but current machine learning approaches face significant challenges in this area compared to biological learning systems. Machine learning algorithms typically trade off retention of previously learned information and adaptation to new or changing data patterns. When continual learning capabilities are absent, algorithms must undergo retraining using the entire data set, an approach that becomes impractical when original training data are unavailable due to storage constraints, financial or computational costs, or privacy restrictions. However, biological animals can learn continually, without experiencing catastrophic forgetting. This paper attempts to build a high-level framework for how animals learn and preserve knowledge by modelling neural components and states that are known to be related to memory consolidation. We focus on three concepts: experience replay, REM sleep, and bilaterality. We propose 4MAS (4 Module Awake/Sleep), a novel macroarchitecture demonstrating how machine learning models might benefit from asymmetric hemispheres, each with their own long- and short-term memory mechanisms, and how a period of sleep between incremental learning tasks might benefit memory consolidation. Finally, we present results showing that our architecture achieves statistical improvements over established generative replay baselines on Split-MNIST (98.3%) and Split-Fashion-MNIST (84.9%), while reducing forgetting by more than half relative to the strongest replay baseline and demonstrating monotonic capacity scaling on Split-CIFAR-100 (29.29%).
发表机构
- Monash University（莫纳什大学）
- CSIRO（联邦科学与工业研究组织）
- Cerenaut（塞雷诺特公司）
机构由 AI 辅助整理，请以论文原文为准。
