少记多留：一种轻量级正则化方法用于基于复习的持续学习
Forget Less, Retain More: A Lightweight Regularizer for Rehearsal-Based Continual Learning
AI总结：
本文提出了一种轻量级正则化方法，用于减少持续学习中的遗忘问题，通过信息最大化策略提升模型性能。
摘要：
深度神经网络在学习新任务时会经历灾难性遗忘，即在训练新任务后，先前任务的性能会下降。这种现象是由于模型倾向于用新信息覆盖之前获得的知识。我们提出了一种新的方法来解决这一挑战，重点在于基于记忆的方法和正则化方法的交集。我们提出了一种正则化策略，称为信息最大化（IM）正则化，用于基于记忆的持续学习方法，该策略仅基于预期的标签分布，因此是类无关的。因此，IM正则化可以被直接集成到各种基于复习的持续学习方法中，减少遗忘并促进更快的收敛。我们的实证验证显示，在各种数据集和不管任务数量的情况下，我们提出的正则化策略都能在极小的计算开销下提升基线性能。IM的轻量级性质使其成为一种实用且可扩展的解决方案，适用于对效率要求极高的现实持续学习场景。最后，我们通过将其应用于视频数据来展示正则化器的数据无关性，视频数据由于其时间结构和更高的内存需求而带来了额外的挑战。尽管存在显著的领域差距，我们的实验显示IM正则化器也提高了视频持续学习方法的性能。
英文摘要：
Deep neural networks suffer from catastrophic forgetting, where performance on previous tasks degrades after training on a new task. This issue arises due to the model's tendency to overwrite previously acquired knowledge with new information. We present a novel approach to address this challenge, focusing on the intersection of memory-based methods and regularization approaches. We formulate a regularization strategy, termed Information Maximization (IM) regularizer, for memory-based continual learning methods, which is based exclusively on the expected label distribution, thus making it class-agnostic. As a consequence, IM regularizer can be directly integrated into various rehearsal-based continual learning methods, reducing forgetting and favoring faster convergence. Our empirical validation shows that, across datasets and regardless of the number of tasks, our proposed regularization strategy consistently improves baseline performance at the expense of a minimal computational overhead. The lightweight nature of IM ensures that it remains a practical and scalable solution, making it applicable to real-world continual learning scenarios where efficiency is paramount. Finally, we demonstrate the data-agnostic nature of our regularizer by applying it to video data, which presents additional challenges due to its temporal structure and higher memory requirements. Despite the significant domain gap, our experiments show that IM regularizer also improves the performance of video continual learning methods.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
