面向预训练模型的类别增量学习高效回放方法
An Efficient Replay for Class-Incremental Learning with Pre-trained Models
AI总结：
该研究针对预训练模型下的类别增量学习灾难性遗忘问题，发现类别中心引导的权重稳态破坏与遗忘相关，提出仅保留单类单样本回放加简单梯度约束的方法，以极低计算成本实现了有竞争力的性能。
摘要：
在常规类别增量学习中，研究人员通常使用样本集作为工具，以避免持续学习过程中的灾难性遗忘。同时，研究人员也注意到了类别增量学习与Oracle训练之间的差异，并尝试进行修正。近年来，研究人员开始利用预训练模型开发类别增量学习算法，取得了显著成果。本文观察到，在类别增量学习中，由各类别中心引导的权重之间的稳态会被打破，这与灾难性遗忘显著相关。基于此，我们提出了一种克服遗忘的新方法。在部分场景下，仅在内存中保留每个类别的单个样本单元用于回放，并施加简单的梯度约束，即可取得非常好的效果。实验结果表明，在预训练模型条件下，我们的方法仅需极低的计算成本，且仅使用交叉熵损失，就能达到具有竞争力的性能。
英文摘要：
In general class-incremental learning, researchers typically use sample sets as a tool to avoid catastrophic forgetting during continuous learning. At the same time, researchers have also noted the differences between class-incremental learning and Oracle training and have attempted to make corrections. In recent years, researchers have begun to develop class-incremental learning algorithms utilizing pre-trained models, achieving significant results. This paper observes that in class-incremental learning, the steady state among the weight guided by each class center is disrupted, which is significantly correlated with catastrophic forgetting. Based on this, we propose a new method to overcoming forgetting . In some cases, by retaining only a single sample unit of each class in memory for replay and applying simple gradient constraints, very good results can be achieved. Experimental results indicate that under the condition of pre-trained models, our method can achieve competitive performance with very low computational cost and by simply using the cross-entropy loss.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
