通过协作学习提升在线持续学习中的可塑性
Improving Plasticity in Online Continual Learning via Collaborative Learning
AI总结：
本文针对在线持续学习中模型可塑性不足的问题，提出协作持续学习（CCL）与蒸馏链（DC）策略，通过协作学习提升新知识获取能力，并在现有方法上显著改善整体性能。
摘要：
在线持续学习（CL）解决的是从连续数据流中学习不断出现的新分类任务的问题。与离线对应方法不同，在在线持续学习中，训练数据只能被看到一次。现有的大多数在线持续学习研究几乎将灾难性遗忘（即模型稳定性）视为唯一的挑战。在本文中，我们认为模型获取新知识的能力（即模型可塑性）是在线持续学习中的另一个挑战。虽然基于回放的策略已被证明在缓解灾难性遗忘方面是有效的，但在提升模型可塑性方面的研究关注仍存在明显不足。为此，我们提出了协作持续学习（CCL），这是一种基于协作学习的策略，用于提升模型获取新概念的能力。此外，我们引入了蒸馏链（DC），这是一种用于促进模型训练的协作学习方案。我们将 CCL-DC 适配到现有的代表性在线持续学习工作中。大量实验表明，即使学习者已经使用最先进的在线持续学习方法进行了良好训练，我们的策略仍然能够显著提升模型可塑性，从而大幅提高整体性能。我们工作的源代码可在 https://github.com/maorong-wang/CCL-DC 获取。
英文摘要：
Online Continual Learning (CL) solves the problem of learning the ever-emerging new classification tasks from a continuous data stream. Unlike its offline counterpart, in online CL, the training data can only be seen once. Most existing online CL research regards catastrophic forgetting (i.e., model stability) as almost the only challenge. In this paper, we argue that the model's capability to acquire new knowledge (i.e., model plasticity) is another challenge in online CL. While replay-based strategies have been shown to be effective in alleviating catastrophic forgetting, there is a notable gap in research attention toward improving model plasticity. To this end, we propose Collaborative Continual Learning (CCL), a collaborative learning based strategy to improve the model's capability in acquiring new concepts. Additionally, we introduce Distillation Chain (DC), a collaborative learning scheme to boost the training of the models. We adapt CCL-DC to existing representative online CL works. Extensive experiments demonstrate that even if the learners are well-trained with state-of-the-art online CL methods, our strategy can still improve model plasticity dramatically, and thereby improve the overall performance by a large margin. The source code of our work is available at https://github.com/maorong-wang/CCL-DC.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
