基于分组和独占稀疏正则化的CNN持续学习
Group and Exclusive Sparse Regularization-based Continual Learning of CNNs
AI总结：
本文提出GESCL方法，通过稳定性与塑性正则化减少持续学习中的灾难性遗忘，提升整体性能。
摘要：
我们提出了一种基于正则化的持续学习（CL）方法，用于固定容量的卷积神经网络（CNN）的持续学习，该方法在连续学习多个任务时不会出现灾难性遗忘问题。这种方法称为分组和独占稀疏持续学习（GESCL），通过引入稳定性正则化项来确保CNN的稳定性，防止被检测为过去任务重要性的滤波器在学习新任务时发生太大偏差。此外，GESCL通过引入塑性正则化项利用CNN的过参数化特性，高效地对网络进行稀疏化，并调整不重要的滤波器使其对未来任务相关。这样做使GESCL在参数和计算上显著少于那些动态扩展网络或记忆过去任务数据的持续学习方法。在流行的持续学习视觉基准测试中，GESCL在整体持续学习性能方面显著优于现有最先进的方法，这在分类准确率以及避免灾难性遗忘方面得到了验证。
英文摘要：
We present a regularization-based approach for continual learning (CL) of fixed capacity convolutional neural networks (CNN) that does not suffer from the problem of catastrophic forgetting when learning multiple tasks sequentially. This method referred to as Group and Exclusive Sparsity based Continual Learning (GESCL) avoids forgetting of previous tasks by ensuring the stability of the CNN via a stability regularization term, which prevents filters detected as important for past tasks to deviate too much when learning a new task. On top of that, GESCL makes the network plastic via a plasticity regularization term that leverage the over-parameterization of CNNs to efficiently sparsify the network and tunes unimportant filters making them relevant for future tasks. Doing so, GESCL deals with significantly less parameters and computation compared to CL approaches that either dynamically expand the network or memorize past tasks' data. Experiments on popular CL vision benchmarks show that GESCL leads to significant improvements over state-of-the-art method in terms of overall CL performance, as measured by classification accuracy as well as in terms of avoiding catastrophic forgetting.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
