面向有限数据的持续学习的无监督回放策略
Unsupervised Replay Strategies for Continual Learning with Limited Data
AI总结：
本文研究无监督“睡眠”回放对有限且不平衡数据下持续学习的影响，发现其能缓解灾难性遗忘并提升先前任务性能，尤其在数据有限时效果显著。
摘要：
人工神经网络（ANNs）在训练数据稀缺或不平衡时性能有限，并在持续学习中面临挑战，例如在新任务训练后遗忘先前学习的数据。相比之下，人脑能够持续学习，并且只需少量示例即可学习。本研究探讨了“睡眠”（一种无监督阶段，结合随机激活与局部 Hebbian 学习规则）对在有限且不平衡的数据集（具体为 MNIST 和 Fashion MNIST）上增量训练的 ANNs 的影响。我们发现，引入睡眠阶段显著提升了在有限数据下训练的模型的准确率。当依次训练少量任务时，睡眠回放不仅挽救了在新任务训练后发生灾难性遗忘的先前学习信息，而且常常提升先前任务的性能，尤其是那些使用有限数据训练的任务。本研究强调了睡眠回放在增强学习效率和促进 ANNs 持续学习中的多方面作用。
英文摘要：
Artificial neural networks (ANNs) show limited performance with scarce or imbalanced training data and face challenges with continuous learning, such as forgetting previously learned data after new tasks training. In contrast, the human brain can learn continuously and from just a few examples. This research explores the impact of 'sleep', an unsupervised phase incorporating stochastic activation with local Hebbian learning rules, on ANNs trained incrementally with limited and imbalanced datasets, specifically MNIST and Fashion MNIST. We discovered that introducing a sleep phase significantly enhanced accuracy in models trained with limited data. When a few tasks were trained sequentially, sleep replay not only rescued previously learned information that had been catastrophically forgetting following new task training but often enhanced performance in prior tasks, especially those trained with limited data. This study highlights the multifaceted role of sleep replay in augmenting learning efficiency and facilitating continual learning in ANNs.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
