语义引导的动态稀疏化用于基于预训练模型的类增量学习
Semantic-Guided Dynamic Sparsification for Pre-Trained Model-based Class-Incremental Learning
AI总结：
本文提出语义引导的动态稀疏化方法，通过动态稀疏化策略减少类间干扰，提升基于预训练模型的类增量学习性能。
摘要：
类增量学习（CIL）要求模型能够持续学习新类别而不遗忘旧类别。一种常见且高效的解决方案是冻结预训练模型并使用轻量级适配器，其参数通常被强制正交以防止跨任务干扰。然而，我们认为这种参数约束方法对可塑性是有害的。为此，我们提出了语义引导的动态稀疏化（SGDS），一种新的方法，通过定向稀疏化来主动引导激活空间。具体来说，SGDS通过促进相似类别共享紧凑的激活子空间来促进知识迁移，同时通过为不相似类别分配不重叠的激活子空间来防止干扰。通过在激活空间中雕刻类别特定的稀疏子空间，SGDS有效缓解了干扰，而无需对参数空间施加刚性约束。在各种基准数据集上的广泛实验表明了SGDS的最先进性能。
英文摘要：
Class-Incremental Learning (CIL) requires a model to continually learn new classes without forgetting old ones. A common and efficient solution freezes a pre-trained model and employs lightweight adapters, whose parameters are often forced to be orthogonal to prevent inter-task interference. However, we argue that this parameter-constraining method is detrimental to plasticity. To this end, we propose Semantic-Guided Dynamic Sparsification (SGDS), a novel method that proactively guides the activation space by governing the orientation and rank of its subspaces through targeted sparsification. Specifically, SGDS promotes knowledge transfer by encouraging similar classes to share a compact activation subspace, while simultaneously preventing interference by assigning non-overlapping activation subspaces to dissimilar classes. By sculpting class-specific sparse subspaces in the activation space, SGDS effectively mitigates interference without imposing rigid constraints on the parameter space. Extensive experiments on various benchmark datasets demonstrate the state-of-the-art performance of SGDS.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
