利用自蒸馏缓解在线类别增量学习中的灾难性遗忘
Reducing Catastrophic Forgetting in Online Class Incremental Learning Using Self-Distillation
AI总结：
针对持续学习中的灾难性遗忘及重放方法的泛化性问题，提出基于浅层输出作教师的自蒸馏方法与易误分类样本优先存储的内存更新策略，在多个数据集上性能优于传统方法。
摘要：
在持续学习中存在严重的灾难性遗忘问题，即模型学习新任务时会遗忘先前的知识。学界已提出多种方法来解决该问题，其中重放方法（即在后续训练中重放先前任务的数据）展现出了不错的准确率，但这类方法受限于有限的内存缓冲区，存在泛化性问题。本文尝试通过自蒸馏获取可迁移知识来解决该问题，具体是将浅层中泛化性较强的输出作为教师信号。此外，当处理大量类别或高难度数据时，存在学习无法收敛、不会出现过拟合的风险，因此我们提出了一种新的内存更新方法，优先存储易被误分类的样本，以实现更高效、更充分的学习。我们在CIFAR10、CIFAR100和MiniimageNet数据集上开展实验，证实所提方法的性能优于传统方法。
英文摘要：
In continual learning, there is a serious problem of catastrophic forgetting, in which previous knowledge is forgotten when a model learns new tasks. Various methods have been proposed to solve this problem. Replay methods which replay data from previous tasks in later training, have shown good accuracy. However, replay methods have a generalizability problem from a limited memory buffer. In this paper, we tried to solve this problem by acquiring transferable knowledge through self-distillation using highly generalizable output in shallow layer as a teacher. Furthermore, when we deal with a large number of classes or challenging data, there is a risk of learning not converging and not experiencing overfitting. Therefore, we attempted to achieve more efficient and thorough learning by prioritizing the storage of easily misclassified samples through a new method of memory update. We confirmed that our proposed method outperformed conventional methods by experiments on CIFAR10, CIFAR100, and MiniimageNet datasets.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
