尽力学习并充分休息以实现持续学习
Do Your Best and Get Enough Rest for Continual Learning
AI总结：
本研究借鉴艾宾浩斯遗忘曲线理论，提出view-batch模型，通过最优回忆间隔重放与单样本自监督学习两种方法，有效缓解持续学习中的灾难性遗忘问题，显著提升了多种现有SOTA方法的性能。
摘要：
根据艾宾浩斯遗忘曲线理论，我们可以通过学习大量数据并保证充足休息来提升记忆保持效果。这意味着为了有效留存新知识，必须对其进行充分学习，并确保足够的休息，从而让大脑能够记忆内容而不遗忘。该理论的核心结论是，一次性学习大量数据后，需要充足的休息才能再次学习相同数据。人类长期记忆保持的这一特性可被有效用于解决神经网络的持续学习问题。长期留存新知识且不发生灾难性遗忘是持续学习的核心难题。\n因此，基于艾宾浩斯理论，我们提出了view-batch（视图批次）模型，该模型通过调整学习调度来优化相同样本重训练之间的回忆间隔。所提出的view-batch模型让网络获得充分休息，从而能以足够长的回忆间隔从相同样本中学习大量知识。为此，我们具体提出了两种方法：1）保证最优回忆间隔的重放方法；2）每次从单个训练样本中获取大量知识的自监督学习方法。我们通过实验证明，本方法的这些实现路径与遗忘曲线理论一致，能够增强长期记忆。在实验中，我们还证明本方法在多种协议和场景下显著提升了多种当前最优的持续学习方法的性能。我们已在https://github.com/hankyul2/ViewBatchModel 开源该项目。
英文摘要：
According to the forgetting curve theory, we can enhance memory retention by learning extensive data and taking adequate rest. This means that in order to effectively retain new knowledge, it is essential to learn it thoroughly and ensure sufficient rest so that our brain can memorize without forgetting. The main takeaway from this theory is that learning extensive data at once necessitates sufficient rest before learning the same data again. This aspect of human long-term memory retention can be effectively utilized to address the continual learning of neural networks. Retaining new knowledge for a long period of time without catastrophic forgetting is the critical problem of continual learning. Therefore, based on Ebbinghaus' theory, we introduce the view-batch model that adjusts the learning schedules to optimize the recall interval between retraining the same samples. The proposed view-batch model allows the network to get enough rest to learn extensive knowledge from the same samples with a recall interval of sufficient length. To this end, we specifically present two approaches: 1) a replay method that guarantees the optimal recall interval, and 2) a self-supervised learning that acquires extensive knowledge from a single training sample at a time. We empirically show that these approaches of our method are aligned with the forgetting curve theory, which can enhance long-term memory. In our experiments, we also demonstrate that our method significantly improves many state-of-the-art continual learning methods in various protocols and scenarios. We open-source this project at https://github.com/hankyul2/ViewBatchModel.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
