arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2206.02577cs.LGcs.AI
收藏

辅助知识对持续学习的影响

Effects of Auxiliary Knowledge on Continual Learning

University of Catania(卡塔尼亚大学);University of Modena and Reggio Emilia(摩德纳雷焦艾米利亚大学)

Giovanni Bellitto, Matteo Pennisi, Simone Palazzo, Lorenzo Bonicelli, Matteo Boschini, Simone Calderara, Concetto Spampinato

摘要:

在持续学习(CL)中,神经网络在分布随时间变化的数据流上进行训练。在这种情况下,主要问题是如何学习新信息而不忘记旧知识(即,灾难性遗忘)。大多数现有的CL方法专注于寻找解决方案,以保存已获得的知识,因此工作在模型的过去。然而,我们认为,由于模型必须不断学习新任务,因此关注现有知识也很重要,这些知识可以改善后续任务的学习。在本文中,我们提出了一个新的,简单的,CL算法,专注于解决当前的任务的方式,可能有利于学习下一个。更具体地说,我们的方法将主要数据流与次要的、多样的和不相关的数据流相结合,网络可以从中提取辅助知识。这从不同的角度帮助模型,因为辅助数据可能包含当前和下一个任务的有用特征,并且传入的任务类可以映射到辅助类。此外,向当前任务添加数据隐含地使分类器更加鲁棒,因为我们正在强制提取更具鉴别力的特征。我们的方法可以优于现有的最先进的模型上最常见的CL图像分类基准。

英文摘要:

In Continual Learning (CL), a neural network is trained on a stream of data whose distribution changes over time. In this context, the main problem is how to learn new information without forgetting old knowledge (i.e., Catastrophic Forgetting). Most existing CL approaches focus on finding solutions to preserve acquired knowledge, so working on the past of the model. However, we argue that as the model has to continually learn new tasks, it is also important to put focus on the present knowledge that could improve following tasks learning. In this paper we propose a new, simple, CL algorithm that focuses on solving the current task in a way that might facilitate the learning of the next ones. More specifically, our approach combines the main data stream with a secondary, diverse and uncorrelated stream, from which the network can draw auxiliary knowledge. This helps the model from different perspectives, since auxiliary data may contain useful features for the current and the next tasks and incoming task classes can be mapped onto auxiliary classes. Furthermore, the addition of data to the current task is implicitly making the classifier more robust as we are forcing the extraction of more discriminative features. Our method can outperform existing state-of-the-art models on the most common CL Image Classification benchmarks.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑