我们提出了一种全新的TACLE（TAsk and CLass-awarE，任务与类别感知）框架，用于解决无样本半监督类别增量学习这一研究较少且颇具挑战性的问题。在该场景下，模型在每个新任务中都需要从少量有标注数据和无标注数据中学习新类别，且无法访问旧类别的样本。除了利用预训练模型的能力外，TACLE还提出了一种新颖的任务自适应阈值，从而在增量学习推进过程中最大化对可用无标注数据的利用率。此外，为了提升每个任务中样本占比不足的类别的性能，我们提出了一种类别感知加权交叉熵损失。我们还利用无标注数据进行分类器对齐，进一步提升了模型性能。在CIFAR10、CIFAR100和ImageNet-Subset100这几个基准数据集上开展的大量实验证明了所提TACLE框架的有效性。我们还进一步展示了其在无标注数据不平衡场景以及每个类别仅一个标注样本的极端场景下的有效性。
英文摘要：
We propose a novel TACLE (TAsk and CLass-awarE) framework to address the relatively unexplored and challenging problem of exemplar-free semi-supervised class incremental learning. In this scenario, at each new task, the model has to learn new classes from both (few) labeled and unlabeled data without access to exemplars from previous classes. In addition to leveraging the capabilities of pre-trained models, TACLE proposes a novel task-adaptive threshold, thereby maximizing the utilization of the available unlabeled data as incremental learning progresses. Additionally, to enhance the performance of the under-represented classes within each task, we propose a class-aware weighted cross-entropy loss. We also exploit the unlabeled data for classifier alignment, which further enhances the model performance. Extensive experiments on benchmark datasets, namely CIFAR10, CIFAR100, and ImageNet-Subset100 demonstrate the effectiveness of the proposed TACLE framework. We further showcase its effectiveness when the unlabeled data is imbalanced and also for the extreme case of one labeled example per class.
Exemplar-free incremental learning is extremely challenging due to inaccessibility of data from old tasks. In this paper, we attempt to exploit the knowledge encoded in a previously trained classification model to handle the catastrophic forgetting problem in continual learning. Specifically, we introduce a so-called knowledge delegator, which is capable of transferring knowledge from the trained model to a randomly re-initialized new model by generating informative samples. Given the previous model only, the delegator is effectively learned using a self-distillation mechanism in a data-free m…
Exemplar-Free Class Incremental Learning via Incremental Representation
Institute of Computing Technology, Chinese Academy of Sciences（中国科学院计算技术研究所）；School of Mathematics and Statistics, Beijing Technology and Business University（北京工商大学数学与统计学院）
Rotation Augmented Distillation for Exemplar-Free Class Incremental Learning with Detailed Analysis
School of Artificial Intelligence, Sun Yat-sen University（中山大学人工智能学院）；Guangdong Key Laboratory of Big Data Analysis and Processing（广东省大数据分析与处理重点实验室）；Key Laboratory of Machine Intelligence and Advanced Computing, Ministry of Education（教育部机器智能与先进计算重点实验室）
Sun Yat-sen University（中山大学）；Key Laboratory of Machine Intelligence and Advanced Computing, Ministry of Education（教育部机器智能与先进计算重点实验室）；Peng Cheng Laboratory（鹏城实验室）
Deep neural network (DNN) suffers from catastrophic forgetting when learning incrementally, which greatly limits its applications. Although maintaining a handful of samples (called `exemplars`) of each task could alleviate forgetting to some extent, existing methods are still limited by the small number of exemplars since these exemplars are too few to carry enough task-specific knowledge, and therefore the forgetting remains. To overcome this problem, we propose to `imagine` diverse counterparts of given exemplars referring to the abundant semantic-irrelevant information from unlabeled data. …
Incremental learning is a form of online learning. Incremental learning can modify the parameters and structure of the deep learning model so that the model does not forget the old knowledge while learning new knowledge. Preventing catastrophic forgetting is the most important task of incremental learning. However, the current incremental learning is often only for one type of input. For example, if the input images are of the same type, the current incremental model can learn new knowledge while not forgetting old knowledge. However, if several categories are added to the input graphics, the …
University of Science and Technology of China（中国科学技术大学）；Institute of Artificial Intelligence, Hefei Comprehensive National Science Center（合肥综合性国家科学中心人工智能研究院）；University of Rochester（罗切斯特大学）
Non-exemplar class-incremental learning is to recognize both the old and new classes when old class samples cannot be saved. It is a challenging task since representation optimization and feature retention can only be achieved under supervision from new classes. To address this problem, we propose a novel self-sustaining representation expansion scheme. Our scheme consists of a structure reorganization strategy that fuses main-branch expansion and side-branch updating to maintain the old features, and a main-branch distillation scheme to transfer the invariant knowledge. Furthermore, a prototy…
Prompt-Based Exemplar Super-Compression and Regeneration for Class-Incremental Learning
Yale University（耶鲁大学）；Johns Hopkins University（约翰霍普金斯大学）；University of Freiburg（弗赖堡大学）；Max Planck Institute for Informatics（马克斯·普朗克信息研究所）；University of Illinois Urbana-Champaign（伊利诺伊大学厄巴纳-香槟分校）
FeTrIL++: Feature Translation for Exemplar-Free Class-Incremental Learning with Hill-Climbing
West University of Timisoara（蒂米什瓦拉西部大学）；Université Paris-Saclay（巴黎-萨克雷大学）；CEA（法国原子能委员会）；LIGM（巴黎路桥学校信息学实验室）；Ecole des Ponts（巴黎路桥学校）；Univ Gustave Eiffel（Gustave Eiffel大学）；CNRS（法国国家科学研究中心）
On the Discrimination and Consistency for Exemplar-Free Class Incremental Learning
The Hong Kong Polytechnic University（香港理工大学）；University College London（伦敦大学学院）；Huazhong University of Science and Technology（华中科技大学）；The University of Queensland（昆士兰大学）
Towards Non-Exemplar Semi-Supervised Class-Incremental Learning
University of Chinese Academy of Sciences（中国科学院大学）；Institute of Automation, Chinese Academy of Sciences（中国科学院自动化研究所）；Centre for Artificial Intelligence and Robotics, Hong Kong Institute of Science and Innovation, Chinese Academy of Sciences（中国科学院香港创新研究院人工智能与机器人中心）
Diffusion Model Meets Non-Exemplar Class-Incremental Learning and Beyond
Beijing National Research Center for Information Science and Technology (BNRist)（北京信息科学与技术国家研究中心（BNRist））；Department of Electronic Engineering, Tsinghua University（清华大学电子工程系）
Boosting the Class-Incremental Learning in 3D Point Clouds via Zero-Collection-Cost Basic Shape Pre-Training
School of Intelligent Engineering and Automation, Beijing University of Posts and Telecommunications（北京邮电大学智能工程与自动化学院）；School of Electronic Engineering, Beijing University of Posts and Telecommunications（北京邮电大学电子工程学院）
Efficient Non-Exemplar Class-Incremental Learning with Retrospective Feature Synthesis
School of Computer Science and Technology, Beijing Institute of Technology（北京理工大学计算机科学与技术学院）；School of Optics and Photonics, Beijing Institute of Technology（北京理工大学光电学院）；School of Medical Technology, Beijing Institute of Technology（北京理工大学医学技术学院）
REAL: Representation Enhanced Analytic Learning for Exemplar-free Class-incremental Learning
Shien-Ming Wu School of Intelligent Engineering（申铭武智能工程学院）；School of Future Technology（未来技术学院）；Department of Electrical and Electronic Engineering（电气与电子工程系）
Northeastern University（东北大学）；Microsoft Research（微软研究院）；City University of New York（纽约城市大学）
In this paper, we address the incremental classifier learning problem, which suffers from catastrophic forgetting. The main reason for catastrophic forgetting is that the past data are not available during learning. Typical approaches keep some exemplars for the past classes and use distillation regularization to retain the classification capability on the past classes and balance the past and new classes. However, there are four main problems with these approaches. First, the loss function is not efficient for classification. Second, there is unbalance problem between the past and new classes…
Non-Exemplar Online Class-incremental Continual Learning via Dual-prototype Self-augment and Refinement
The Hong Kong Polytechnic University（香港理工大学）；The Hong Kong Polytechnic University Shenzhen Research Institute（香港理工大学深圳研究院）；Huazhong University of Science and Technology（华中科技大学）
Exemplar-based class-incremental learning is to recognize new classes while not forgetting old ones, whose samples can only be saved in limited memory. The ratio fluctuation of new samples to old exemplars, which is caused by the variation of memory capacity at different environments, will bring challenges to stabilize the incremental optimization process. To address this problem, we propose a novel self-paced imbalance rectification scheme, which dynamically maintains the incremental balance during the representation learning phase. Specifically, our proposed scheme consists of a frequency co…
Incremental learning aims to enable machine learning models to continuously acquire new knowledge given new classes, while maintaining the knowledge already learned for old classes. Saving a subset of training samples of previously seen classes in the memory and replaying them during new training phases is proven to be an efficient and effective way to fulfil this aim. It is evident that the larger number of exemplars the model inherits the better performance it can achieve. However, finding a trade-off between the model performance and the number of samples to save for each class is still an …
EFC++: Elastic Feature Consolidation with Prototype Re-balancing for Cold Start Exemplar-free Incremental Learning
Communication Center (MICC), University of Florence, Italy Tomaso Trinci. ORCID: 0000-0002-4052-1930 Global Optimization Laboratory, University of Florence, Italy Albin Soutif - Cormerais. ORCID: 0009-0008-1564-2029 LAMP Team, Computer Vision Center, Barcelona, Spain Joost van de Weijer. ORCID: 0000-0002-9656-9706 LAMP Team, Computer Vision Center, Universitat Aut\` o noma de Barcelona, Spain Andrew D. Bagdanov. ORCID: 0000-0001-6408-7043 Media Integration；Communication Center (MICC), University of Florence, Italy Equal contribution
Tianjin University（天津大学）；Max Planck Institute for Informatics（马克斯·普朗克信息学研究所）；Singapore Management University（新加坡管理大学）
Multi-Class Incremental Learning (MCIL) aims to learn new concepts by incrementally updating a model trained on previous concepts. However, there is an inherent trade-off to effectively learning new concepts without catastrophic forgetting of previous ones. To alleviate this issue, it has been proposed to keep around a few examples of the previous concepts but the effectiveness of this approach heavily depends on the representativeness of these examples. This paper proposes a novel and automatic framework we call mnemonics, where we parameterize exemplars and make them optimizable in an end-to…
Seoul National University（首尔大学）；NAVER AI Lab（NAVER AI实验室）；NAVER Clova
This paper introduces a solid state-of-the-art baseline for a class-incremental semantic segmentation (CISS) problem. While the recent CISS algorithms utilize variants of the knowledge distillation (KD) technique to tackle the problem, they failed to fully address the critical challenges in CISS causing the catastrophic forgetting; the semantic drift of the background class and the multi-label prediction issue. To better address these challenges, we propose a new method, dubbed SSUL-M (Semantic Segmentation with Unknown Label with Memory), by carefully combining techniques tailored for semanti…
College of Computer Science and Technology, Zhejiang University（浙江大学计算机科学与技术学院）；Shanghai Institute for Advanced Study, Zhejiang University（浙江大学上海高等研究院）
With the memory-resource-limited constraints, class-incremental learning (CIL) usually suffers from the "catastrophic forgetting" problem when updating the joint classification model on the arrival of newly added classes. To cope with the forgetting problem, many CIL methods transfer the knowledge of old classes by preserving some exemplar samples into the size-constrained memory buffer. To utilize the memory buffer more efficiently, we propose to keep more auxiliary low-fidelity exemplar samples rather than the original real high-fidelity exemplar samples. Such a memory-efficient exemplar pre…
PASS++: A Dual Bias Reduction Framework for Non-Exemplar Class-Incremental Learning
Centre for Artificial Intelligence and Robotics, Hong Kong Institute of Science and Innovation, Chinese Academy of Sciences（中国科学院香港创新研究院人工智能与机器人中心）；Institute of Automation of Chinese Academy of Sciences（中国科学院自动化研究所）；School of Artificial Intelligence, University of Chinese Academy of Sciences（中国科学院大学人工智能学院）
Task-recency bias strikes back: Adapting covariances in Exemplar-Free Class Incremental Learning
IDEAS NCBR；Warsaw University of Technology（华沙工业大学）；Gdańsk University of Technology（格但斯克工业大学）；Tooploox；Autonomous University of Barcelona（巴塞罗那自治大学）；Computer Vision Center（计算机视觉中心）
Real-world applications require the classification model to adapt to new classes without forgetting old ones. Correspondingly, Class-Incremental Learning (CIL) aims to train a model with limited memory size to meet this requirement. Typical CIL methods tend to save representative exemplars from former classes to resist forgetting, while recent works find that storing models from history can substantially boost the performance. However, the stored models are not counted into the memory budget, which implicitly results in unfair comparisons. We find that when counting the model size into the tot…
University at Buffalo（布法罗大学）；Penn State Erie（宾州州立大学伊利分校）
Exemplar-free Class-incremental Learning (CIL) is a challenging problem because rehearsing data from previous phases is strictly prohibited, causing catastrophic forgetting of Deep Neural Networks (DNNs). In this paper, we present iVoro, a holistic framework for CIL, derived from computational geometry. We found Voronoi Diagram (VD), a classical model for space subdivision, is especially powerful for solving the CIL problem, because VD itself can be constructed favorably in an incremental manner -- the newly added sites (classes) will only affect the proximate classes, making the non-contiguou…
Seoul National University（首尔国立大学）；Sungkyunkwan University（成均馆大学）
We consider class incremental learning (CIL) problem, in which a learning agent continuously learns new classes from incrementally arriving training data batches and aims to predict well on all the classes learned so far. The main challenge of the problem is the catastrophic forgetting, and for the exemplar-memory based CIL methods, it is generally known that the forgetting is commonly caused by the classification score bias that is injected due to the data imbalance between the new classes and the old classes (in the exemplar-memory). While several methods have been proposed to correct such s…
CEAT: Continual Expansion and Absorption Transformer for Non-Exemplar Class-Incremental Learning
School of Software Engineering, Xi’an Jiaotong University（西安交通大学软件学院）；Institute of Artificial Intelligence and Robotics, Xi’an Jiaotong University（西安交通大学人工智能与机器人研究所）
Indian Institute of Science（印度科学理工学院）；BITS Pilani（比尔拉技术与科学学院皮拉尼校区）
Pose estimation is the task of locating keypoints for an object of interest in an image. Animal Pose estimation is more challenging than estimating human pose due to high inter and intra class variability in animals. Existing works solve this problem for a fixed set of predefined animal categories. Models trained on such sets usually do not work well with new animal categories. Retraining the model on new categories makes the model overfit and leads to catastrophic forgetting. Thus, in this work, we propose a novel problem of "Incremental Learning for Animal Pose Estimation". Our method uses a…
Cross-modal retrieval deals with retrieving relevant items from one modality, when provided with a search query from another modality. Hashing techniques, where the data is represented as binary bits have specifically gained importance due to the ease of storage, fast computations and high accuracy. In real world, the number of data categories is continuously increasing, which requires algorithms capable of handling this dynamic scenario. In this work, we propose a novel incremental cross-modal hashing algorithm termed "iCMH", which can adapt itself to handle incoming data of new categories. T…
