理解基于复习的类增量学习中的不平衡遗忘
Understanding Imbalanced Forgetting in Rehearsal-Based Class-Incremental Learning
AI总结：
本文研究了基于复习的类增量学习中不平衡遗忘现象，通过分析梯度层面的干扰源，提出三种系数以预测类别的遗忘程度，揭示了自我诱导干扰对遗忘的影响。
摘要：
神经网络在类增量学习（CIL）设置中面临灾难性遗忘问题。复习（重新播放部分过去的样本）是已知的缓解策略。然而，最近的结果表明，尽管复习分配平衡，某些类别的遗忘程度仍然显著更大。尽管其相关性，这种不平衡遗忘现象仍被忽视。本文展示了基于复习的CIL中不平衡遗忘系统性且严重，并进行了深入研究。具体而言，我们从原理分析中构建了三个最后一层系数，捕捉不同梯度层面的干扰源，影响每个过去类别在增量步骤中的表现。然后证明，这些系数共同可靠地预测了过去类别在该步骤结束时的遗忘排名。虽然预测性能本身不足以确立因果性，但这些结果支持将这些系数解释为可能的机制解释，将训练过程中最后一层梯度层面的相互作用与类别层面的遗忘结果联系起来。值得注意的是，一个捕捉自我诱导干扰的系数成为最强预测因子，受控实验提供了支持这一系数受新类别干扰系数影响的证据。总体而言，我们的发现提供了有价值的见解，并建议通过减少识别的干扰源在类别间的差异来缓解不平衡遗忘。
英文摘要：
Neural networks suffer from catastrophic forgetting in class-incremental learning (CIL) settings. Rehearsal$\unicode{x2013}$replaying a subset of past samples$\unicode{x2013}$is a well-established mitigation strategy. However, recent results suggest that, despite balanced rehearsal allocation, some classes are forgotten substantially more than others. Despite its relevance, this imbalanced forgetting phenomenon remains underexplored. This work shows that imbalanced forgetting arises systematically and severely in rehearsal-based CIL and investigates it extensively. Specifically, we construct, from a principled analysis, three last-layer coefficients that capture different gradient-level sources of interference affecting each past class during an incremental step. We then demonstrate that, together, they reliably predict how past classes will rank in terms of forgetting at the end of that step. While predictive performance alone does not establish causality, these results support the interpretation of the coefficients as a plausible mechanistic account linking last-layer gradient-level interactions during training to class-level forgetting outcomes. Notably, one coefficient$\unicode{x2013}$capturing self-induced interference$\unicode{x2013}$emerges as the strongest predictor, with controlled experiments providing evidence consistent with this coefficient being influenced by the new-class interference coefficient. Overall, our findings provide valuable insights and suggest promising directions for mitigating imbalanced forgetting by reducing class-wise disparities in the identified sources of interference.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
