Incremental learning is a form of online learning. Incremental learning can modify the parameters and structure of the deep learning model so that the model does not forget the old knowledge while learning new knowledge. Preventing catastrophic forgetting is the most important task of incremental learning. However, the current incremental learning is often only for one type of input. For example, if the input images are of the same type, the current incremental model can learn new knowledge while not forgetting old knowledge. However, if several categories are added to the input graphics, the current model will not be able to deal with it correctly, and the accuracy will drop significantly. Therefore, this paper proposes a kind of incremental method, which adjusts the parameters of the model by identifying the prototype vector and increasing the distance of the vector, so that the model can learn new knowledge without catastrophic forgetting. Experiments show the effectiveness of our proposed method.
Seoul National University（首尔大学）；NAVER AI Lab（NAVER AI实验室）；NAVER Clova
This paper introduces a solid state-of-the-art baseline for a class-incremental semantic segmentation (CISS) problem. While the recent CISS algorithms utilize variants of the knowledge distillation (KD) technique to tackle the problem, they failed to fully address the critical challenges in CISS causing the catastrophic forgetting; the semantic drift of the background class and the multi-label prediction issue. To better address these challenges, we propose a new method, dubbed SSUL-M (Semantic Segmentation with Unknown Label with Memory), by carefully combining techniques tailored for semanti…
CFSSeg: Closed-Form Solution for Class-Incremental Semantic Segmentation of 2D Images and 3D Point Clouds
Central South University（中南大学）；The Hong Kong University of Science and Technology (Guangzhou)（香港科技大学（广州））；South China University of Technology（华南理工大学）；Peking University（北京大学）；Shanghai Artificial Intelligence Laboratory（上海人工智能实验室）；Zhengzhou University（郑州大学）
Zhejiang University（浙江大学）；Zhejiang Lab（之江实验室）；University of California, Los Angeles（加州大学洛杉矶分校）
Conventional detection networks usually need abundant labeled training samples, while humans can learn new concepts incrementally with just a few examples. This paper focuses on a more challenging but realistic class-incremental few-shot object detection problem (iFSD). It aims to incrementally transfer the model for novel objects from only a few annotated samples without catastrophically forgetting the previously learned ones. To tackle this problem, we propose a novel method LEAST, which can transfer with Less forgetting, fEwer training resources, And Stronger Transfer capability. Specifical…
Computer Vision Center UAB（巴塞罗那自治大学计算机视觉中心）；Amazon Web Services（亚马逊云科技）；UCLA（加州大学洛杉矶分校）
Multi-task learns multiple tasks, while sharing knowledge and computation among them. However, it suffers from catastrophic forgetting of previous knowledge when learned incrementally without access to the old data. Most existing object detectors are domain-specific and static, while some are learned incrementally but only within a single domain. Training an object detector incrementally across various domains has rarely been explored. In this work, we propose three incremental learning scenarios across various domains and categories for object detection. To mitigate catastrophic forgetting, a…
Exemplar-free incremental learning is extremely challenging due to inaccessibility of data from old tasks. In this paper, we attempt to exploit the knowledge encoded in a previously trained classification model to handle the catastrophic forgetting problem in continual learning. Specifically, we introduce a so-called knowledge delegator, which is capable of transferring knowledge from the trained model to a randomly re-initialized new model by generating informative samples. Given the previous model only, the delegator is effectively learned using a self-distillation mechanism in a data-free m…
Exemplar-Free Class Incremental Learning via Incremental Representation
Institute of Computing Technology, Chinese Academy of Sciences（中国科学院计算技术研究所）；School of Mathematics and Statistics, Beijing Technology and Business University（北京工商大学数学与统计学院）
Orthogonal Knowledge Refreshing for Domain-Incremental Object Detection
Institute of Information Engineering, Chinese Academy of Sciences（中国科学院信息工程研究所）；Nankai University（南开大学）；School of Cyber Security, University of Chinese Academy of Sciences（中国科学院大学网络空间安全学院）；Tsinghua University（清华大学）；Harbin Institute of Technology（哈尔滨工业大学）
Rotation Augmented Distillation for Exemplar-Free Class Incremental Learning with Detailed Analysis
School of Artificial Intelligence, Sun Yat-sen University（中山大学人工智能学院）；Guangdong Key Laboratory of Big Data Analysis and Processing（广东省大数据分析与处理重点实验室）；Key Laboratory of Machine Intelligence and Advanced Computing, Ministry of Education（教育部机器智能与先进计算重点实验室）
Sun Yat-sen University（中山大学）；Key Laboratory of Machine Intelligence and Advanced Computing, Ministry of Education（教育部机器智能与先进计算重点实验室）；Peng Cheng Laboratory（鹏城实验室）
Deep neural network (DNN) suffers from catastrophic forgetting when learning incrementally, which greatly limits its applications. Although maintaining a handful of samples (called `exemplars`) of each task could alleviate forgetting to some extent, existing methods are still limited by the small number of exemplars since these exemplars are too few to carry enough task-specific knowledge, and therefore the forgetting remains. To overcome this problem, we propose to `imagine` diverse counterparts of given exemplars referring to the abundant semantic-irrelevant information from unlabeled data. …
University of Science and Technology of China（中国科学技术大学）；Institute of Artificial Intelligence, Hefei Comprehensive National Science Center（合肥综合性国家科学中心人工智能研究院）；University of Rochester（罗切斯特大学）
Non-exemplar class-incremental learning is to recognize both the old and new classes when old class samples cannot be saved. It is a challenging task since representation optimization and feature retention can only be achieved under supervision from new classes. To address this problem, we propose a novel self-sustaining representation expansion scheme. Our scheme consists of a structure reorganization strategy that fuses main-branch expansion and side-branch updating to maintain the old features, and a main-branch distillation scheme to transfer the invariant knowledge. Furthermore, a prototy…
Prompt-Based Exemplar Super-Compression and Regeneration for Class-Incremental Learning
Yale University（耶鲁大学）；Johns Hopkins University（约翰霍普金斯大学）；University of Freiburg（弗赖堡大学）；Max Planck Institute for Informatics（马克斯·普朗克信息研究所）；University of Illinois Urbana-Champaign（伊利诺伊大学厄巴纳-香槟分校）
FeTrIL++: Feature Translation for Exemplar-Free Class-Incremental Learning with Hill-Climbing
West University of Timisoara（蒂米什瓦拉西部大学）；Université Paris-Saclay（巴黎-萨克雷大学）；CEA（法国原子能委员会）；LIGM（巴黎路桥学校信息学实验室）；Ecole des Ponts（巴黎路桥学校）；Univ Gustave Eiffel（Gustave Eiffel大学）；CNRS（法国国家科学研究中心）
On the Discrimination and Consistency for Exemplar-Free Class Incremental Learning
The Hong Kong Polytechnic University（香港理工大学）；University College London（伦敦大学学院）；Huazhong University of Science and Technology（华中科技大学）；The University of Queensland（昆士兰大学）
Towards Non-Exemplar Semi-Supervised Class-Incremental Learning
University of Chinese Academy of Sciences（中国科学院大学）；Institute of Automation, Chinese Academy of Sciences（中国科学院自动化研究所）；Centre for Artificial Intelligence and Robotics, Hong Kong Institute of Science and Innovation, Chinese Academy of Sciences（中国科学院香港创新研究院人工智能与机器人中心）
Diffusion Model Meets Non-Exemplar Class-Incremental Learning and Beyond
Beijing National Research Center for Information Science and Technology (BNRist)（北京信息科学与技术国家研究中心（BNRist））；Department of Electronic Engineering, Tsinghua University（清华大学电子工程系）
Boosting the Class-Incremental Learning in 3D Point Clouds via Zero-Collection-Cost Basic Shape Pre-Training
School of Intelligent Engineering and Automation, Beijing University of Posts and Telecommunications（北京邮电大学智能工程与自动化学院）；School of Electronic Engineering, Beijing University of Posts and Telecommunications（北京邮电大学电子工程学院）
University of Southern California（南加州大学）；Arizona State University（亚利桑那州立大学）；Samsung Research America（三星美国研究院）
Deep neural networks (DNNs) often suffer from "catastrophic forgetting" during incremental learning (IL) --- an abrupt degradation of performance on the original set of classes when the training objective is adapted to a newly added set of classes. Existing IL approaches tend to produce a model that is biased towards either the old classes or new classes, unless with the help of exemplars of the old data. To address this issue, we propose a class-incremental learning paradigm called Deep Model Consolidation (DMC), which works well even when the original training data is not available. The idea…
Efficient Non-Exemplar Class-Incremental Learning with Retrospective Feature Synthesis
School of Computer Science and Technology, Beijing Institute of Technology（北京理工大学计算机科学与技术学院）；School of Optics and Photonics, Beijing Institute of Technology（北京理工大学光电学院）；School of Medical Technology, Beijing Institute of Technology（北京理工大学医学技术学院）
REAL: Representation Enhanced Analytic Learning for Exemplar-free Class-incremental Learning
Shien-Ming Wu School of Intelligent Engineering（申铭武智能工程学院）；School of Future Technology（未来技术学院）；Department of Electrical and Electronic Engineering（电气与电子工程系）
Northeastern University（东北大学）；Microsoft Research（微软研究院）；City University of New York（纽约城市大学）
In this paper, we address the incremental classifier learning problem, which suffers from catastrophic forgetting. The main reason for catastrophic forgetting is that the past data are not available during learning. Typical approaches keep some exemplars for the past classes and use distillation regularization to retain the classification capability on the past classes and balance the past and new classes. However, there are four main problems with these approaches. First, the loss function is not efficient for classification. Second, there is unbalance problem between the past and new classes…
Non-Exemplar Online Class-incremental Continual Learning via Dual-prototype Self-augment and Refinement
The Hong Kong Polytechnic University（香港理工大学）；The Hong Kong Polytechnic University Shenzhen Research Institute（香港理工大学深圳研究院）；Huazhong University of Science and Technology（华中科技大学）
University of Toronto（多伦多大学）；Vector Institute（向量学院）；Google Research（谷歌研究院）
We introduce Exemplar VAEs, a family of generative models that bridge the gap between parametric and non-parametric, exemplar based generative models. Exemplar VAE is a variant of VAE with a non-parametric prior in the latent space based on a Parzen window estimator. To sample from it, one first draws a random exemplar from a training set, then stochastically transforms that exemplar into a latent code and a new observation. We propose retrieval augmented training (RAT) as a way to speed up Exemplar VAE training by using approximate nearest neighbor search in the latent space to define a lower…
Exemplar-based class-incremental learning is to recognize new classes while not forgetting old ones, whose samples can only be saved in limited memory. The ratio fluctuation of new samples to old exemplars, which is caused by the variation of memory capacity at different environments, will bring challenges to stabilize the incremental optimization process. To address this problem, we propose a novel self-paced imbalance rectification scheme, which dynamically maintains the incremental balance during the representation learning phase. Specifically, our proposed scheme consists of a frequency co…
Incremental learning aims to enable machine learning models to continuously acquire new knowledge given new classes, while maintaining the knowledge already learned for old classes. Saving a subset of training samples of previously seen classes in the memory and replaying them during new training phases is proven to be an efficient and effective way to fulfil this aim. It is evident that the larger number of exemplars the model inherits the better performance it can achieve. However, finding a trade-off between the model performance and the number of samples to save for each class is still an …
EFC++: Elastic Feature Consolidation with Prototype Re-balancing for Cold Start Exemplar-free Incremental Learning
Communication Center (MICC), University of Florence, Italy Tomaso Trinci. ORCID: 0000-0002-4052-1930 Global Optimization Laboratory, University of Florence, Italy Albin Soutif - Cormerais. ORCID: 0009-0008-1564-2029 LAMP Team, Computer Vision Center, Barcelona, Spain Joost van de Weijer. ORCID: 0000-0002-9656-9706 LAMP Team, Computer Vision Center, Universitat Aut\` o noma de Barcelona, Spain Andrew D. Bagdanov. ORCID: 0000-0001-6408-7043 Media Integration；Communication Center (MICC), University of Florence, Italy Equal contribution
Tianjin University（天津大学）；Max Planck Institute for Informatics（马克斯·普朗克信息学研究所）；Singapore Management University（新加坡管理大学）
Multi-Class Incremental Learning (MCIL) aims to learn new concepts by incrementally updating a model trained on previous concepts. However, there is an inherent trade-off to effectively learning new concepts without catastrophic forgetting of previous ones. To alleviate this issue, it has been proposed to keep around a few examples of the previous concepts but the effectiveness of this approach heavily depends on the representativeness of these examples. This paper proposes a novel and automatic framework we call mnemonics, where we parameterize exemplars and make them optimizable in an end-to…
College of Computer Science and Technology, Zhejiang University（浙江大学计算机科学与技术学院）；Shanghai Institute for Advanced Study, Zhejiang University（浙江大学上海高等研究院）
With the memory-resource-limited constraints, class-incremental learning (CIL) usually suffers from the "catastrophic forgetting" problem when updating the joint classification model on the arrival of newly added classes. To cope with the forgetting problem, many CIL methods transfer the knowledge of old classes by preserving some exemplar samples into the size-constrained memory buffer. To utilize the memory buffer more efficiently, we propose to keep more auxiliary low-fidelity exemplar samples rather than the original real high-fidelity exemplar samples. Such a memory-efficient exemplar pre…
PASS++: A Dual Bias Reduction Framework for Non-Exemplar Class-Incremental Learning
Centre for Artificial Intelligence and Robotics, Hong Kong Institute of Science and Innovation, Chinese Academy of Sciences（中国科学院香港创新研究院人工智能与机器人中心）；Institute of Automation of Chinese Academy of Sciences（中国科学院自动化研究所）；School of Artificial Intelligence, University of Chinese Academy of Sciences（中国科学院大学人工智能学院）
Task-recency bias strikes back: Adapting covariances in Exemplar-Free Class Incremental Learning
IDEAS NCBR；Warsaw University of Technology（华沙工业大学）；Gdańsk University of Technology（格但斯克工业大学）；Tooploox；Autonomous University of Barcelona（巴塞罗那自治大学）；Computer Vision Center（计算机视觉中心）
Real-world applications require the classification model to adapt to new classes without forgetting old ones. Correspondingly, Class-Incremental Learning (CIL) aims to train a model with limited memory size to meet this requirement. Typical CIL methods tend to save representative exemplars from former classes to resist forgetting, while recent works find that storing models from history can substantially boost the performance. However, the stored models are not counted into the memory budget, which implicitly results in unfair comparisons. We find that when counting the model size into the tot…
University at Buffalo（布法罗大学）；Penn State Erie（宾州州立大学伊利分校）
Exemplar-free Class-incremental Learning (CIL) is a challenging problem because rehearsing data from previous phases is strictly prohibited, causing catastrophic forgetting of Deep Neural Networks (DNNs). In this paper, we present iVoro, a holistic framework for CIL, derived from computational geometry. We found Voronoi Diagram (VD), a classical model for space subdivision, is especially powerful for solving the CIL problem, because VD itself can be constructed favorably in an incremental manner -- the newly added sites (classes) will only affect the proximate classes, making the non-contiguou…
