面向无回放增量学习的高效扩展与基于梯度的任务推断
Efficient Expansion and Gradient Based Task Inference for Replay Free Incremental Learning
AI总结：
本文提出一种基于滤波器和通道扩展的无回放持续学习方法，在先前任务参数上按任务复杂度自适应扩展模型以提升知识迁移，并通过熵加权数据增强与伪标签梯度实现鲁棒任务ID预测，在TIL、CIL和生成式持续学习中取得最先进结果。
摘要：
本文提出了一种简单但高效的基于扩展的持续学习模型。近期的特征变换、掩码和基于分解的方法虽然高效，但它们仅在全局或共享参数上扩展模型。因此，这些方法未能充分利用先前学到的信息，因为同一个任务特定参数会遗忘早期知识。因此，这些方法表现出有限的迁移学习能力。此外，这些模型大多对所有任务保持恒定的参数增长，而不考虑任务复杂度。我们的工作提出了一种简单的基于滤波器和通道扩展的方法，该方法在先前任务参数上扩展模型，而不仅仅是在全局参数上扩展。因此，它能够在不遗忘的情况下充分利用所有先前学到的信息，从而实现更好的知识迁移。我们提出的模型中的增长率是任务复杂度的函数；因此对于简单任务，模型的参数增长较小，而对于复杂任务，模型需要更多参数来适应当前任务。近期基于扩展的模型在任务增量学习（TIL）中展现出有前景的结果。然而，对于类增量学习（CIL），任务ID的预测是一个关键挑战；因此，随着任务数量的增加，它们的结果会迅速下降。在这项工作中，我们提出了一种鲁棒的任务预测方法，该方法利用熵加权数据增强和使用伪标签的模型梯度。我们在TIL、CIL和生成式持续学习设置下，在多个数据集和架构上评估了我们的模型。所提出的方法在所有这些设置中都展现了最先进的结果。我们广泛的消融研究证明了所提出组件的有效性。
英文摘要：
This paper proposes a simple but highly efficient expansion-based model for continual learning. The recent feature transformation, masking and factorization-based methods are efficient, but they grow the model only over the global or shared parameter. Therefore, these approaches do not fully utilize the previously learned information because the same task-specific parameter forgets the earlier knowledge. Thus, these approaches show limited transfer learning ability. Moreover, most of these models have constant parameter growth for all tasks, irrespective of the task complexity. Our work proposes a simple filter and channel expansion based method that grows the model over the previous task parameters and not just over the global parameter. Therefore, it fully utilizes all the previously learned information without forgetting, which results in better knowledge transfer. The growth rate in our proposed model is a function of task complexity; therefore for a simple task, the model has a smaller parameter growth while for complex tasks, the model requires more parameters to adapt to the current task. Recent expansion based models show promising results for task incremental learning (TIL). However, for class incremental learning (CIL), prediction of task id is a crucial challenge; hence, their results degrade rapidly as the number of tasks increase. In this work, we propose a robust task prediction method that leverages entropy weighted data augmentations and the models gradient using pseudo labels. We evaluate our model on various datasets and architectures in the TIL, CIL and generative continual learning settings. The proposed approach shows state-of-the-art results in all these settings. Our extensive ablation studies show the efficacy of the proposed components.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
