预测样本对灾难性遗忘的易感性
Predicting the Susceptibility of Examples to Catastrophic Forgetting
AI总结：
本文发现学习速度快的样本更不易被遗忘，据此提出基于学习速度的回放采样策略SBS，可广泛提升持续学习基准上的性能。
摘要：
灾难性遗忘——神经网络在学习新信息时遗忘先前所学数据的倾向——仍然是持续学习中的核心挑战。在本工作中，我们采用行为学方法，观察到学习速度与遗忘之间的联系：学习得更快的样本更不容易被遗忘。聚焦于基于回放的持续学习，我们表明回放缓冲区的组成——具体而言，它包含的是快速学习还是缓慢学习的样本——对遗忘有显著影响。受这一见解启发，我们提出了基于速度的采样（Speed-Based Sampling，SBS），这是一种简单而通用的策略，根据样本的学习速度来选择回放样本。SBS 可以轻松集成到现有的基于缓冲区的持续学习方法中，并在广泛的竞争性持续学习基准上提升性能，推进了最先进的结果。我们的发现强调了在设计持续学习算法时考虑遗忘动态的价值。
英文摘要：
Catastrophic forgetting - the tendency of neural networks to forget previously learned data when learning new information - remains a central challenge in continual learning. In this work, we adopt a behavioral approach, observing a connection between learning speed and forgetting: examples learned more quickly are less prone to forgetting. Focusing on replay-based continual learning, we show that the composition of the replay buffer - specifically, whether it contains quickly or slowly learned examples - has a significant effect on forgetting. Motivated by this insight, we introduce Speed-Based Sampling (SBS), a simple yet general strategy that selects replay examples based on their learning speed. SBS integrates easily into existing buffer-based methods and improves performance across a wide range of competitive continual learning benchmarks, advancing state-of-the-art results. Our findings underscore the value of accounting for the forgetting dynamics when designing continual learning algorithms.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
