第二届大湾区国际算法大赛序列任务持续学习赛道解决方案
The Solution for the sequential task continual learning track of the 2nd Greater Bay Area International Algorithm Competition
AI总结：
提出一种无数据、基于参数隔离的持续学习算法，通过任务子空间、推理任务识别、梯度补充、自适应评分和掩码压缩，在任务与域增量设置下取得优异表现并获大赛二等奖。
摘要：
本文介绍了我们为第二届大湾区国际算法大赛序列任务持续学习赛道开发的一种无数据、基于参数隔离的持续学习算法。该方法在网络的卷积层和线性层内为每个任务学习独立的参数子空间，并在第一个任务后冻结批归一化层。具体而言，对于所有域共享一个分类头的域增量设置，我们在第一个任务完成后冻结共享分类头，从而有效解决灾难性遗忘问题。此外，面对域增量设置中不提供任务身份的挑战，我们设计了一种推理任务身份策略，为每个样本选择合适的掩码矩阵。我们还引入了梯度补充策略，以增强当前任务未被选择参数的重要性，促进新任务的学习。我们还实现了一种自适应重要性评分策略，可动态调整参数量，以优化单任务性能，同时减少参数使用。此外，考虑到存储空间和推理时间的限制，我们设计了掩码矩阵压缩策略，以节省存储空间并提高掩码矩阵的加密和解密速度。我们的方法不需要扩展核心网络，也不需要使用外部辅助网络或数据，并且在任务增量和域增量设置下均表现良好。该解决方案最终在比赛中获得二等奖。
英文摘要：
This paper presents a data-free, parameter-isolation-based continual learning algorithm we developed for the sequential task continual learning track of the 2nd Greater Bay Area International Algorithm Competition. The method learns an independent parameter subspace for each task within the network's convolutional and linear layers and freezes the batch normalization layers after the first task. Specifically, for domain incremental setting where all domains share a classification head, we freeze the shared classification head after first task is completed, effectively solving the issue of catastrophic forgetting. Additionally, facing the challenge of domain incremental settings without providing a task identity, we designed an inference task identity strategy, selecting an appropriate mask matrix for each sample. Furthermore, we introduced a gradient supplementation strategy to enhance the importance of unselected parameters for the current task, facilitating learning for new tasks. We also implemented an adaptive importance scoring strategy that dynamically adjusts the amount of parameters to optimize single-task performance while reducing parameter usage. Moreover, considering the limitations of storage space and inference time, we designed a mask matrix compression strategy to save storage space and improve the speed of encryption and decryption of the mask matrix. Our approach does not require expanding the core network or using external auxiliary networks or data, and performs well under both task incremental and domain incremental settings. This solution ultimately won a second-place prize in the competition.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
