用于记忆重放类增量学习的平衡破坏-重建动力学
Balanced Destruction-Reconstruction Dynamics for Memory-replay Class Incremental Learning
AI总结：
针对记忆重放类增量学习中的灾难性遗忘问题，提出平衡破坏-重建模块（BDR），通过动态操纵梯度平衡新旧样本贡献，显著提升现有最先进方法的性能与泛化能力。
摘要：
类增量学习（CIL）旨在用新类样本增量更新已训练模型（可塑性），同时保留已学能力（稳定性）。为解决该目标中最具挑战性的灾难性遗忘问题，主流范式是记忆重放CIL，通过重放保存在内存中的少量旧类样本来巩固旧知识。尽管有效，记忆重放CIL中固有的破坏-重建动力学是一个内在限制：若旧知识被严重破坏，将很难重建无损的对应知识。我们的理论分析表明，通过平衡当前阶段样本与内存中保存样本的贡献，可以有效缓解旧知识的破坏。受此理论发现启发，我们提出了一种用于记忆重放CIL的新型平衡破坏-重建模块（BDR），通过降低旧知识的最大破坏程度来实现更好的知识重建。具体而言，为实现旧知识与新类间的更好平衡，所提出的BDR模块考虑了两个因素：不同类间训练状态的差异，以及当前阶段与内存样本的数量不平衡。通过基于这些因素在训练期间动态操纵梯度，BDR能有效缓解知识破坏并改善知识重建。在一系列CIL基准上的大量实验表明，作为轻量级即插即用模块，BDR能显著提升现有最先进方法的性能并具有良好的泛化能力。
英文摘要：
Class incremental learning (CIL) aims to incrementally update a trained model with the new classes of samples (plasticity) while retaining previously learned ability (stability). To address the most challenging issue in this goal, i.e., catastrophic forgetting, the mainstream paradigm is memory-replay CIL, which consolidates old knowledge by replaying a small number of old classes of samples saved in the memory. Despite effectiveness, the inherent destruction-reconstruction dynamics in memory-replay CIL are an intrinsic limitation: if the old knowledge is severely destructed, it will be quite hard to reconstruct the lossless counterpart. Our theoretical analysis shows that the destruction of old knowledge can be effectively alleviated by balancing the contribution of samples from the current phase and those saved in the memory. Motivated by this theoretical finding, we propose a novel Balanced Destruction-Reconstruction module (BDR) for memory-replay CIL, which can achieve better knowledge reconstruction by reducing the degree of maximal destruction of old knowledge. Specifically, to achieve a better balance between old knowledge and new classes, the proposed BDR module takes into account two factors: the variance in training status across different classes and the quantity imbalance of samples from the current phase and memory. By dynamically manipulating the gradient during training based on these factors, BDR can effectively alleviate knowledge destruction and improve knowledge reconstruction. Extensive experiments on a range of CIL benchmarks have shown that as a lightweight plug-and-play module, BDR can significantly improve the performance of existing state-of-the-art methods with good generalization.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
