使用自适应分类准则缓解任务增量持续学习中的灾难性遗忘
Mitigating Catastrophic Forgetting in Task-Incremental Continual Learning with Adaptive Classification Criterion
AI总结：
本文提出一种带有自适应分类准则的监督对比学习持续学习框架（SCCL），通过对比损失学习表示并保存少量样本作为分类准则，结合关系蒸馏与记忆回放缓解灾难性遗忘，实现了优于基线的最先进性能。
摘要：
任务增量持续学习是指在一系列任务中持续训练模型，同时克服灾难性遗忘（CF）问题。该问题的出现是因为为了学习新任务而遗忘了已学到的表示，且决策边界遭到破坏。以往的研究大多考虑如何恢复已学任务的表示。很少有人考虑为新表示调整决策边界，在本文中，我们提出了一个带有自适应分类准则的监督对比学习持续学习框架（SCCL）。在我们的方法中，使用对比损失直接学习不同任务的表示，并保存有限数量的数据样本作为分类准则。在推理期间，将保存的数据样本输入当前模型以获得更新的表示，并使用k近邻模块进行分类。通过这种方式，可扩展的模型能够利用保存样本的自适应准则来解决已学任务。为了缓解CF，我们进一步使用逐实例关系蒸馏正则化项和记忆回放模块来保留先前任务的信息。实验表明，SCCL实现了最先进的性能，并且与分类基线相比具有更强的克服CF的能力。
英文摘要：
Task-incremental continual learning refers to continually training a model in a sequence of tasks while overcoming the problem of catastrophic forgetting (CF). The issue arrives for the reason that the learned representations are forgotten for learning new tasks, and the decision boundary is destructed. Previous studies mostly consider how to recover the representations of learned tasks. It is seldom considered to adapt the decision boundary for new representations and in this paper we propose a Supervised Contrastive learning framework with adaptive classification criterion for Continual Learning (SCCL), In our method, a contrastive loss is used to directly learn representations for different tasks and a limited number of data samples are saved as the classification criterion. During inference, the saved data samples are fed into the current model to obtain updated representations, and a k Nearest Neighbour module is used for classification. In this way, the extensible model can solve the learned tasks with adaptive criteria of saved samples. To mitigate CF, we further use an instance-wise relation distillation regularization term and a memory replay module to maintain the information of previous tasks. Experiments show that SCCL achieves state-of-the-art performance and has a stronger ability to overcome CF compared with the classification baselines.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
