基于标签生成的类别增量学习
Class-Incremental Learning based on Label Generation
AI总结：
针对预训练语言模型在类别增量学习（CIL）中存在灾难性遗忘的问题，本文提出将CIL建模为持续标签生成问题的VAG方法，利用词汇稀疏性与标签语义伪样本提升效果，性能显著优于基线。
摘要：
尽管预训练语言模型已取得巨大成功，但将这类模型用于持续学习仍是一项挑战，尤其是在类别增量学习（class-incremental learning，CIL）场景下，会面临灾难性遗忘（catastrophic forgetting，CF）问题。本文研究发现：若将CIL建模为持续标签生成问题，可大幅降低CF，且能更好地保留预训练模型的可泛化表征。据此我们提出一种新的CIL方法VAG，该方法还利用词汇稀疏性聚焦生成过程，并借助标签语义构造伪回放样本。实验结果表明，VAG的性能大幅优于各基线模型。
英文摘要：
Despite the great success of pre-trained language models, it is still a challenge to use these models for continual learning, especially for the class-incremental learning (CIL) setting due to catastrophic forgetting (CF). This paper reports our finding that if we formulate CIL as a continual label generation problem, CF is drastically reduced and the generalizable representations of pre-trained models can be better retained. We thus propose a new CIL method (VAG) that also leverages the sparsity of vocabulary to focus the generation and creates pseudo-replay samples by using label semantics. Experimental results show that VAG outperforms baselines by a large margin.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
