arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2104.13614cs.CVcs.LG
收藏

在持续学习中利用所有先前的特征提取器保持早期知识

Preserving Earlier Knowledge in Continual Learning with the Help of All Previous Feature Extractors

Sun Yat-sen University(中山大学);Key Laboratory of Machine Intelligence and Advanced Computing, MOE(机器智能与先进计算教育部重点实验室)

Zhuoyun Li, Changhong Zhong, Sijia Liu, Ruixuan Wang, Wei-Shi Zheng

摘要:

随着时间的推移对新知识的持续学习是智能系统识别越来越多类别的对象的一种期望能力。如果没有或只有非常有限的旧数据存储,智能系统在学习新知识时往往会灾难性地忘记以前学习过的旧知识。最近,已经提出了各种方法来缓解灾难性遗忘问题。然而,早期学到的旧知识通常比最近学到的知识保存得更少。为了减少遗忘,特别是早期学习的旧知识,提高整体持续学习性能,我们提出了一个简单而有效的融合机制,包括所有以前学习的特征提取到智能模型。此外,每次学习一组新的类时,将新的特征提取器包含到模型中,并且还应用特征提取器修剪以防止整个模型大小快速增长。在多分类任务上的实验表明,该方法能有效地减少旧知识的遗忘,达到最佳的持续学习性能。

英文摘要:

Continual learning of new knowledge over time is one desirable capability for intelligent systems to recognize more and more classes of objects. Without or with very limited amount of old data stored, an intelligent system often catastrophically forgets previously learned old knowledge when learning new knowledge. Recently, various approaches have been proposed to alleviate the catastrophic forgetting issue. However, old knowledge learned earlier is commonly less preserved than that learned more recently. In order to reduce the forgetting of particularly earlier learned old knowledge and improve the overall continual learning performance, we propose a simple yet effective fusion mechanism by including all the previously learned feature extractors into the intelligent model. In addition, a new feature extractor is included to the model when learning a new set of classes each time, and a feature extractor pruning is also applied to prevent the whole model size from growing rapidly. Experiments on multiple classification tasks show that the proposed approach can effectively reduce the forgetting of old knowledge, achieving state-of-the-art continual learning performance.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑