可扩展子空间集成用于基于预训练模型的类增量学习
Expandable Subspace Ensemble for Pre-Trained Model-Based Class-Incremental Learning
AI总结：
本文提出可扩展子空间集成（EASE）方法，通过为每个新任务训练轻量级适配器创建任务特定子空间，并利用语义引导的原型补充策略合成旧类新特征，在不遗忘旧知识的前提下高效更新模型，在七个基准数据集上达到最新性能。
摘要：
类增量学习（CIL）要求学习系统在不遗忘旧知识的情况下持续学习新类别。尽管预训练模型（PTMs）在类增量学习中表现出强大的性能，但一个关键问题仍然存在：学习新类别往往会导致旧类别的覆盖。对网络的过度修改会导致遗忘，而最小程度的调整又会导致对新类别拟合不足。因此，需要找到一种在不损害先前知识的情况下高效更新模型的方法。在本文中，我们提出了用于基于预训练模型的类增量学习的可扩展子空间集成（EASE）方法。为了实现无冲突的模型更新，我们为每个新任务训练一个独立的轻量级适配器模块，旨在创建任务特定的子空间。这些适配器跨越一个高维特征空间，使得能够在多个子空间中进行联合决策。随着数据的演变，扩展的子空间使得旧类别的分类器与新阶段的子空间不兼容。相应地，我们设计了一种语义引导的原型补充策略，该策略在不使用任何旧类别实例的情况下合成旧类别的新特征。在七个基准数据集上的大量实验验证了EASE的最新性能。代码可在 https://github.com/sun-hailong/CVPR24-Ease 获取。
英文摘要：
Class-Incremental Learning (CIL) requires a learning system to continually learn new classes without forgetting. Despite the strong performance of Pre-Trained Models (PTMs) in CIL, a critical issue persists: learning new classes often results in the overwriting of old ones. Excessive modification of the network causes forgetting, while minimal adjustments lead to an inadequate fit for new classes. As a result, it is desired to figure out a way of efficient model updating without harming former knowledge. In this paper, we propose ExpAndable Subspace Ensemble (EASE) for PTM-based CIL. To enable model updating without conflict, we train a distinct lightweight adapter module for each new task, aiming to create task-specific subspaces. These adapters span a high-dimensional feature space, enabling joint decision-making across multiple subspaces. As data evolves, the expanding subspaces render the old class classifiers incompatible with new-stage spaces. Correspondingly, we design a semantic-guided prototype complement strategy that synthesizes old classes' new features without using any old class instance. Extensive experiments on seven benchmark datasets verify EASE's state-of-the-art performance. Code is available at: https://github.com/sun-hailong/CVPR24-Ease
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
