减缓持续学习中的遗忘
Slowing Down Forgetting in Continual Learning
AI总结：
本文提出名为ReCL的持续学习框架，利用基于梯度的神经网络的隐式偏置在间隔最大化点重建旧任务数据并与当前数据结合，有效减缓灾难性遗忘，在多种场景、数据集和架构上取得显著性能提升。
摘要：
持续学习（CL）中的一个常见挑战是灾难性遗忘，即在学习了新的额外任务后，旧任务上的性能会下降。在本文中，我们提出了一个名为ReCL的新框架来减缓CL中的遗忘。我们的框架利用了基于梯度的神经网络的隐式偏置，由于这种偏置，这些网络会收敛到间隔最大化点。这样的收敛点使我们能够从先前的任务中重建旧数据，然后将其与当前训练数据相结合。我们的框架很灵活，可以应用于现有的最先进的CL方法之上。我们进一步通过大量实验展示了该框架带来的性能提升，包括两个具有挑战性的CL场景（类增量和域增量学习）、不同的数据集（MNIST、CIFAR10、TinyImagenet）以及不同的网络架构。在所有实验中，我们发现ReCL带来了巨大的性能提升。据我们所知，我们的框架是第一个通过将CL中的模型作为其自身的记忆缓冲区来解决灾难性遗忘的框架。
英文摘要：
A common challenge in continual learning (CL) is catastrophic forgetting, where the performance on old tasks drops after new, additional tasks are learned. In this paper, we propose a novel framework called ReCL to slow down forgetting in CL. Our framework exploits an implicit bias of gradient-based neural networks due to which these converge to margin maximization points. Such convergence points allow us to reconstruct old data from previous tasks, which we then combine with the current training data. Our framework is flexible and can be applied on top of existing, state-of-the-art CL methods. We further demonstrate the performance gain from our framework across a large series of experiments, including two challenging CL scenarios (class incremental and domain incremental learning), different datasets (MNIST, CIFAR10, TinyImagenet), and different network architectures. Across all experiments, we find large performance gains through ReCL. To the best of our knowledge, our framework is the first to address catastrophic forgetting by leveraging models in CL as their own memory buffers.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
