唤醒-睡眠整合学习
Wake-Sleep Consolidated Learning
AI总结：
本文提出唤醒-睡眠整合学习WSCL，通过模拟人脑唤醒与睡眠阶段中的记忆巩固和做梦机制，在持续视觉分类任务上显著超越基线方法，并验证了做梦对正向前向迁移的关键作用。
摘要：
我们提出唤醒-睡眠整合学习（Wake-Sleep Consolidated Learning, WSCL），这是一种利用互补学习系统理论和人脑唤醒-睡眠阶段的学习策略，旨在提升深度神经网络在持续学习场景下视觉分类任务的性能。我们的方法通过不同唤醒和睡眠阶段之间的同步进行持续学习。在唤醒阶段，模型接收感官输入并调整其表征，通过动态参数冻结机制确保稳定性，并将情景记忆存储在短期临时记忆中（类似于海马体中所发生的过程）。在睡眠阶段，训练过程分为NREM和REM两个阶段。在NREM阶段，模型利用来自短期记忆和长期记忆的重放样本巩固突触权重，并激活突触可塑性机制，强化重要连接并削弱不重要的连接。在REM阶段，模型接触此前未见过的逼真视觉感官体验，并激活做梦过程，使模型能够探索潜在特征空间，从而为未来知识准备突触。我们在三个基准数据集上评估了所提方法的有效性：CIFAR-10、Tiny-ImageNet和FG-ImageNet。在所有情况下，我们的方法均优于基线和先前工作，在持续视觉分类任务上取得了显著的性能提升。此外，我们证明了所有处理阶段的有效性以及做梦对于实现正向前向迁移的重要性。
英文摘要：
We propose Wake-Sleep Consolidated Learning (WSCL), a learning strategy leveraging Complementary Learning System theory and the wake-sleep phases of the human brain to improve the performance of deep neural networks for visual classification tasks in continual learning settings. Our method learns continually via the synchronization between distinct wake and sleep phases. During the wake phase, the model is exposed to sensory input and adapts its representations, ensuring stability through a dynamic parameter freezing mechanism and storing episodic memories in a short-term temporary memory (similarly to what happens in the hippocampus). During the sleep phase, the training process is split into NREM and REM stages. In the NREM stage, the model's synaptic weights are consolidated using replayed samples from the short-term and long-term memory and the synaptic plasticity mechanism is activated, strengthening important connections and weakening unimportant ones. In the REM stage, the model is exposed to previously-unseen realistic visual sensory experience, and the dreaming process is activated, which enables the model to explore the potential feature space, thus preparing synapses to future knowledge. We evaluate the effectiveness of our approach on three benchmark datasets: CIFAR-10, Tiny-ImageNet and FG-ImageNet. In all cases, our method outperforms the baselines and prior work, yielding a significant performance gain on continual visual classification tasks. Furthermore, we demonstrate the usefulness of all processing stages and the importance of dreaming to enable positive forward transfer.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
