Class-incremental learning of deep networks sequentially increases the number of classes to be classified. During training, the network has only access to data of one task at a time, where each task contains several classes. In this setting, networks suffer from catastrophic forgetting which refers to the drastic drop in performance on previous tasks. The vast majority of methods have studied this scenario for classification networks, where for each new task the classification layer of the network must be augmented with additional weights to make room for the newly added classes. Embedding networks have the advantage that new classes can be naturally included into the network without adding new weights. Therefore, we study incremental learning for embedding networks. In addition, we propose a new method to estimate the drift, called semantic drift, of features and compensate for it without the need of any exemplars. We approximate the drift of previous tasks based on the drift that is experienced by current task data. We perform experiments on fine-grained datasets, CIFAR100 and ImageNet-Subset. We demonstrate that embedding networks suffer significantly less from catastrophic forgetting. We outperform existing methods which do not require exemplars and obtain competitive results compared to methods which store exemplars. Furthermore, we show that our proposed SDC when combined with existing methods to prevent forgetting consistently improves results.
Heterogeneous Forgetting Compensation for Class-Incremental Learning
State Key Laboratory of Robotics, Shenyang Institute of AutomationChinese Academy of Sciences（中国科学院沈阳自动化研究所机器人学国家重点实验室）；Institutes for Robotics and Intelligent ManufacturingChinese Academy of Sciences（中国科学院机器人与智能制造研究所）；University of Chinese Academy of Sciences（中国科学院大学）；South China University of Technology（华南理工大学）
Like humans, deep networks have been shown to learn better when samples are organized and introduced in a meaningful order or curriculum. Conventional curriculum learning schemes introduce samples in their order of difficulty. This forces models to begin learning from a subset of the available data while adding the external overhead of evaluating the difficulty of samples. In this work, we propose Learning with Incremental Labels and Adaptive Compensation (LILAC), a two-phase method that incrementally increases the number of unique output labels rather than the difficulty of samples while cons…
Gradient-Semantic Compensation for Incremental Semantic Segmentation
Shenyang Institute of Automation, Chinese Academy of Sciences（中国科学院沈阳自动化研究所）；Institutes for Robotics and Intelligent Manufacturing, Chinese Academy of Sciences（中国科学院机器人与智能制造创新研究院）；University of Chinese Academy of Sciences（中国科学院大学）；Nanyang Technological University（南洋理工大学）
Continual Adapter Tuning with Semantic Shift Compensation for Class-Incremental Learning
HUST AI & Visual Learning Lab (HAIV Lab)（华中科技大学人工智能与视觉学习实验室）；Huazhong University of Science and Technology (HUST)（华中科技大学）；Department of Computer Science（计算机科学系）；Boston University（波士顿大学）；ByteDance, Inc.（字节跳动公司）
Modern machine learning systems need to be able to cope with constantly arriving and changing data. Two main areas of research dealing with such scenarios are continual learning and data stream mining. Continual learning focuses on accumulating knowledge and avoiding forgetting, assuming information once learned should be stored. Data stream mining focuses on adaptation to concept drift and discarding outdated information, assuming that only the most recent data is relevant. While these two areas are mainly being developed in separation, they offer complementary views on the problem of learnin…
Given a large unlabeled set of images, how to efficiently and effectively group them into clusters based on extracted visual representations remains a challenging problem. To address this problem, we propose a convolutional neural network (CNN) to jointly solve clustering and representation learning in an iterative manner. In the proposed method, given an input image set, we first randomly pick k samples and extract their features as initial cluster centroids using the proposed CNN with an initial model pre-trained from the ImageNet dataset. Mini-batch k-means is then performed to assign clust…
WESTA College, Southwest University（西南大学西塔学院）；College of Artificial Intelligence, Southwest University（西南大学人工智能学院）
Sensor drift is a long-existing unpredictable problem that deteriorates the performance of gaseous substance recognition, calling for an antidrift domain adaptation algorithm. However, the prerequisite for traditional methods to achieve fine results is to have data from both nondrift distributions (source domain) and drift distributions (target domain) for domain alignment, which is usually unrealistic and unachievable in real-life scenarios. To compensate for this, in this paper, deep learning based on a target-domain-free domain adaptation convolutional neural network (TDACNN) is proposed. T…
University of Chinese Academy of Science（中国科学院大学）；Institute of Automation, Chinese Academy of Sciences (CAS)（中国科学院自动化研究所）
Class incremental learning(CIL) has attracted much attention, but most existing related works focus on fine-tuning the entire representation model, which inevitably results in much catastrophic forgetting. In the contrast, with a semantic-rich pre-trained representation model, parameter-additional-tuning (PAT) only changes very few parameters to learn new visual concepts. Recent studies have proved that PAT-based CIL can naturally avoid fighting against forgetting by replaying or distilling like most of the existing methods. However, we find that PAT-based CIL still faces serious semantic drif…
Dual Compensation Residual Networks for Class Imbalanced Learning
Institute of Computing Technology (ICT), Chinese Academy of Sciences (CAS)（中国科学院计算技术研究所）；University of the Chinese Academy of Sciences（中国科学院大学）；School of Computer Science and Technology, University of the Chinese Academy of Sciences（中国科学院大学计算机科学与技术学院）
Scalable Analytic Classifiers with Associative Drift Compensation for Class-Incremental Learning of Vision Transformers
School of Systems Science, Beijing Normal University, Beijing 100875, China（北京师范大学系统科学学院）；School of Artificial Intelligence, Anhui University, Hefei 230601, China（安徽大学人工智能学院）；Faculty of Informatics, Universita' Della Svizzera Italiana, Lugano, Switzerland（瑞士意大利大学信息学院）
ICICLE: Interpretable Class Incremental Continual Learning
Jagiellonian University（雅盖隆大学）；Ardigen SA（阿迪根公司）；Autonomous University of Barcelona（巴塞罗那自治大学）；Computer Vision Center（计算机视觉中心）；IDEAS NCBR（IDEAS国家计算智能研究中心）
Exemplar-based class-incremental learning is to recognize new classes while not forgetting old ones, whose samples can only be saved in limited memory. The ratio fluctuation of new samples to old exemplars, which is caused by the variation of memory capacity at different environments, will bring challenges to stabilize the incremental optimization process. To address this problem, we propose a novel self-paced imbalance rectification scheme, which dynamically maintains the incremental balance during the representation learning phase. Specifically, our proposed scheme consists of a frequency co…
Ain Shams University（艾因夏姆斯大学）；Delft University of Technology（代尔夫特理工大学）
In this paper, an approach for reducing the drift in monocular visual odometry algorithms is proposed based on a feedforward neural network. A visual odometry algorithm computes the incremental motion of the vehicle between the successive camera frames, then integrates these increments to determine the pose of the vehicle. The proposed neural network reduces the errors in the pose estimation of the vehicle which results from the inaccuracies in features detection and matching, camera intrinsic parameters, and so on. These inaccuracies are propagated to the motion estimation of the vehicle caus…
Mixture of Noise for Pre-Trained Model-Based Class-Incremental Learning
School of Artificial Intelligence, OPtics and ElectroNics, Northwestern Polytechnical University（人工智能学院、光学与电子学院、西北工业大学）；Institute of Artificial Intelligence (TeleAI) of China Telecom（中国电信人工智能研究院）；School of Computer Science & School of Artificial Intelligence, Shanghai Jiao Tong University（计算机科学学院及人工智能学院、上海交通大学）；University of Hong Kong（香港大学）
University of Chinese Academy of Sciences（中国科学院大学）；Xiamen University（厦门大学）；Peng Cheng Laboratory（鹏城实验室）
Few-shot class-incremental learning (FSCIL), which targets at continuously expanding model's representation capacity under few supervisions, is an important yet challenging problem. On the one hand, when fitting new tasks (novel classes), features trained on old tasks (old classes) could significantly drift, causing catastrophic forgetting. On the other hand, training the large amount of model parameters with few-shot novel-class examples leads to model over-fitting. In this paper, we propose a learnable expansion-and-compression network (LEC-Net), with the aim to simultaneously solve catastro…
EFC++: Elastic Feature Consolidation with Prototype Re-balancing for Cold Start Exemplar-free Incremental Learning
Communication Center (MICC), University of Florence, Italy Tomaso Trinci. ORCID: 0000-0002-4052-1930 Global Optimization Laboratory, University of Florence, Italy Albin Soutif - Cormerais. ORCID: 0009-0008-1564-2029 LAMP Team, Computer Vision Center, Barcelona, Spain Joost van de Weijer. ORCID: 0000-0002-9656-9706 LAMP Team, Computer Vision Center, Universitat Aut\` o noma de Barcelona, Spain Andrew D. Bagdanov. ORCID: 0000-0001-6408-7043 Media Integration；Communication Center (MICC), University of Florence, Italy Equal contribution
GR4CIL: Gap-compensated Routing for CLIP-based Class Incremental Learning
Department of COMP The Hong Kong Polytechnic University（计算机与信息科学系，香港理工大学）；Department of COMP/LSGI The Hong Kong Polytechnic University（计算机与信息科学系/LSGI，香港理工大学）
Hohai University（河海大学）；Singapore University of Technology and Design（新加坡科技设计大学）；University of Electronic Science and Technology of China（电子科技大学）；Shanghai AI Laboratory（上海人工智能实验室）；Nanjing University（南京大学）
Incremental few-shot semantic segmentation (IFSS) targets at incrementally expanding model's capacity to segment new class of images supervised by only a few samples. However, features learned on old classes could significantly drift, causing catastrophic forgetting. Moreover, few samples for pixel-level segmentation on new classes lead to notorious overfitting issues in each learning session. In this paper, we explicitly represent class-based knowledge for semantic segmentation as a category embedding and a hyper-class embedding, where the former describes exclusive semantical properties, and…
Shenyang Institute of Automation, Chinese Academy of Sciences（中国科学院沈阳自动化研究所）；Institutes for Robotics and Intelligent Manufacturing, Chinese Academy of Sciences（中国科学院机器人与智能制造研究院）；University of Chinese Academy of Sciences（中国科学院大学）；Northeastern University（东北大学）
3D object classification has attracted appealing attentions in academic researches and industrial applications. However, most existing methods need to access the training data of past 3D object classes when facing the common real-world scenario: new classes of 3D objects arrive in a sequence. Moreover, the performance of advanced approaches degrades dramatically for past learned classes (i.e., catastrophic forgetting), due to the irregular and redundant geometric structures of 3D point cloud data. To address these challenges, we propose a new Incremental 3D Object Learning (i.e., I3DOL) model,…
Efficient Non-Exemplar Class-Incremental Learning with Retrospective Feature Synthesis
School of Computer Science and Technology, Beijing Institute of Technology（北京理工大学计算机科学与技术学院）；School of Optics and Photonics, Beijing Institute of Technology（北京理工大学光电学院）；School of Medical Technology, Beijing Institute of Technology（北京理工大学医学技术学院）
Tokyo Institute of Technology（东京工业大学）；Olympus Corporation（奥林巴斯公司）
Pipe inspection is a critical task for many industries and infrastructure of a city. The 3D information of a pipe can be used for revealing the deformation of the pipe surface and position of the camera during the inspection. In this paper, we propose a 3D pipe reconstruction system using sequential images captured by a monocular endoscopic camera. Our work extends a state-of-the-art incremental Structure-from-Motion (SfM) method to incorporate prior constraints given by the target shape into bundle adjustment (BA). Using this constraint, we can minimize the scale-drift that is the general pro…
Shenyang Institute of Automation, Chinese Academy of Sciences（中国科学院沈阳自动化研究所）；University of Chinese Academy of Sciences（中国科学院大学）；Northwestern University（西北大学）；Sony AI（索尼人工智能研究院）；Nanjing University of Science and Technology（南京理工大学）；ETH Zurich（苏黎世联邦理工学院）
3D object recognition has successfully become an appealing research topic in the real-world. However, most existing recognition models unreasonably assume that the categories of 3D objects cannot change over time in the real-world. This unrealistic assumption may result in significant performance degradation for them to learn new classes of 3D objects consecutively, due to the catastrophic forgetting on old learned classes. Moreover, they cannot explore which 3D geometric characteristics are essential to alleviate the catastrophic forgetting on old classes of 3D objects. To tackle the above ch…
