Few-Shot Class-Incremental Learning with Prior Knowledge
East China Normal University（华东师范大学）；DiDi（滴滴）；Shanghai Jiao Tong University（上海交通大学）；Tsinghua Berkeley Shenzhen Institute（清华伯克利深圳学院）；Toronto Metropolitan University（多伦多都会大学）
为解决小样本类别增量学习（FSCIL）中的灾难性遗忘与过拟合问题，以往研究主要聚焦于在增量阶段保留对旧知识的记忆，而预训练模型对增量学习效果的塑造作用常被低估。为此，为提升预训练模型的泛化能力，我们提出“基于先验知识学习（Learning with Prior Knowledge, LwPK）”方法，从后续增量类的少量无标注数据中引入近乎零成本的先验知识。我们对无标注的增量类样本进行聚类以生成伪标签，再将其与有标注的基类样本联合训练，为新旧类数据有效分配嵌入空间。实验结果表明，LwPK有效增强了模型抵御灾难性遗忘的能力，基于经验风险最小化与类别距离度量的理论分析也验证了其作用原理。LwPK的源代码已公开于：https://github.com/StevenJ308/LwPK。
英文摘要：
To tackle the issues of catastrophic forgetting and overfitting in few-shot class-incremental learning (FSCIL), previous work has primarily concentrated on preserving the memory of old knowledge during the incremental phase. The role of pre-trained model in shaping the effectiveness of incremental learning is frequently underestimated in these studies. Therefore, to enhance the generalization ability of the pre-trained model, we propose Learning with Prior Knowledge (LwPK) by introducing nearly free prior knowledge from a few unlabeled data of subsequent incremental classes. We cluster unlabeled incremental class samples to produce pseudo-labels, then jointly train these with labeled base class samples, effectively allocating embedding space for both old and new class data. Experimental results indicate that LwPK effectively enhances the model resilience against catastrophic forgetting, with theoretical analysis based on empirical risk minimization and class distance measurement corroborating its operational principles. The source code of LwPK is publicly available at: \url{https://github.com/StevenJ308/LwPK}.
Korea Advanced Institute of Science and Technology (KAIST)（韩国科学技术院（KAIST））
Inspired by Regularized Lottery Ticket Hypothesis (RLTH), which hypothesizes that there exist smooth (non-binary) subnetworks within a dense network that achieve the competitive performance of the dense network, we propose a few-shot class incremental learning (FSCIL) method referred to as \emph{Soft-SubNetworks (SoftNet)}. Our objective is to learn a sequence of sessions incrementally, where each session only includes a few training instances per class while preserving the knowledge of the previously learned ones. SoftNet jointly learns the model weights and adaptive non-binary soft masks at …
Few-shot class-incremental learning (FSCIL) has addressed challenging real-world scenarios where unseen novel classes continually arrive with few samples. In these scenarios, it is required to develop a model that recognizes the novel classes without forgetting prior knowledge. In other words, FSCIL aims to maintain the base performance and improve the novel performance simultaneously. However, there is little study to investigate the two performances separately. In this paper, we first decompose the entire model into four types of parameters and demonstrate that the tendency of the two perfor…
Ulsan National Institute of Science and Technology（蔚山国立科学技术院）；Korea Advanced Institute of Science and Technology（韩国科学技术院）
Learning novel concepts while preserving prior knowledge is a long-standing challenge in machine learning. The challenge gets greater when a novel task is given with only a few labeled examples, a problem known as incremental few-shot learning. We propose XtarNet, which learns to extract task-adaptive representation (TAR) for facilitating incremental few-shot learning. The method utilizes a backbone network pretrained on a set of base categories while also employing additional modules that are meta-trained across episodes. Given a new task, the novel feature extracted from the meta-trained mod…
Humans are capable of learning new concepts from only a few (labeled) exemplars, incrementally and continually. This happens within the context that we can differentiate among the exemplars, and between the exemplars and large amounts of other data (unlabeled and labeled). This suggests, in human learning, supervised learning of concepts based on exemplars takes place within the larger context of contrastive self-supervised learning (CSSL) based on unlabeled and labeled data. We discuss extending CSSL (1) to be based mainly on exemplars and only secondly on data augmentation, and (2) to apply …
Few-shot Class-incremental Learning for Cross-domain Disease Classification
Shenzhen Institute of Advanced Technology, Chinese Academy of Sciences（中国科学院深圳先进技术研究院）；University of Chinese Academy of Sciences（中国科学院大学）；Pengcheng Laboratory（鹏城实验室）
The ability to incrementally learn new classes is crucial to the development of real-world artificial intelligence systems. In this paper, we focus on a challenging but practical few-shot class-incremental learning (FSCIL) problem. FSCIL requires CNN models to incrementally learn new classes from very few labelled samples, without forgetting the previously learned ones. To address this problem, we represent the knowledge using a neural gas (NG) network, which can learn and preserve the topology of the feature manifold formed by different classes. On this basis, we propose the TOpology-Preservi…
This paper considers incremental few-shot learning, which requires a model to continually recognize new categories with only a few examples provided. Our study shows that existing methods severely suffer from catastrophic forgetting, a well-known problem in incremental learning, which is aggravated due to data scarcity and imbalance in the few-shot setting. Our analysis further suggests that to prevent catastrophic forgetting, actions need to be taken in the primitive stage -- the training of base classes instead of later few-shot learning sessions. Therefore, we propose to search for flat loc…
Few-shot class-incremental learning (FSCIL) aims to design machine learning algorithms that can continually learn new concepts from a few data points, without forgetting knowledge of old classes. The difficulty lies in that limited data from new classes not only lead to significant overfitting issues but also exacerbate the notorious catastrophic forgetting problems. Moreover, as training data come in sequence in FSCIL, the learned classifier can only provide discriminative information in individual sessions, while FSCIL requires all classes to be involved for evaluation. In this paper, we add…
Gaussian processes (GPs) are non-parametric, flexible, models that work well in many tasks. Combining GPs with deep learning methods via deep kernel learning (DKL) is especially compelling due to the strong representational power induced by the network. However, inference in GPs, whether with or without DKL, can be computationally challenging on large datasets. Here, we propose GP-Tree, a novel method for multi-class classification with Gaussian processes and DKL. We develop a tree-based hierarchical model in which each internal node of the tree fits a GP to the data using the Pólya Gamma augm…
Arizona State University（亚利桑那州立大学）；City University of Hong Kong（香港城市大学）
The ability to incrementally learn new classes is vital to all real-world artificial intelligence systems. A large portion of high-impact applications like social media, recommendation systems, E-commerce platforms, etc. can be represented by graph models. In this paper, we investigate the challenging yet practical problem, Graph Few-shot Class-incremental (Graph FCL) problem, where the graph model is tasked to classify both newly encountered classes and previously learned classes. Towards that purpose, we put forward a Graph Pseudo Incremental Learning paradigm by sampling tasks recurrently f…
Few-shot class incremental learning -- the problem of updating a trained classifier to discriminate among an expanded set of classes with limited labeled data -- is a key challenge for machine learning systems deployed in non-stationary environments. Existing approaches to the problem rely on complex model architectures and training procedures that are difficult to tune and re-use. In this paper, we present an extremely simple approach that enables the use of ordinary logistic regression classifiers for few-shot incremental learning. The key to this approach is a new family of subspace regular…
Incremental learning attempts to develop a classifier which learns continuously from a stream of data segregated into different classes. Deep learning approaches suffer from catastrophic forgetting when learning classes incrementally, while most incremental learning approaches require a large amount of training data per class. We examine the problem of incremental learning using only a few training examples, referred to as Few-Shot Incremental Learning (FSIL). To solve this problem, we propose a novel approach inspired by the concept learning model of the hippocampus and the neocortex that rep…
Nanjing University（南京大学）；Hikvision Research Institute（海康威视研究院）
New classes arise frequently in our ever-changing world, e.g., emerging topics in social media and new types of products in e-commerce. A model should recognize new classes and meanwhile maintain discriminability over old classes. Under severe circumstances, only limited novel instances are available to incrementally update the model. The task of recognizing few-shot new classes without forgetting old classes is called few-shot class-incremental learning (FSCIL). In this work, we propose a new paradigm for FSCIL based on meta-learning by LearnIng Multi-phase Incremental Tasks (LIMIT), which sy…
Huazhong University of Science and Technology（华中科技大学）
Different from fine-tuning models pre-trained on a large-scale dataset of preset classes, class-incremental learning (CIL) aims to recognize novel classes over time without forgetting pre-trained classes. However, a given model will be challenged by test images with finer-grained classes, e.g., a basenji is at most recognized as a dog. Such images form a new training set (i.e., support set) so that the incremental model is hoped to recognize a basenji (i.e., query) as a basenji next time. This paper formulates such a hybrid natural problem of coarse-to-fine few-shot (C2FS) recognition as a CIL…
Few-Shot Domain Incremental Learning via Continual Vision-Language Consolidation
School of Computer Science and IT, Adelaide University, Australia（阿德莱德大学计算机科学与信息技术学院，澳大利亚）；Faculty of Engineering and IT, University of Technology Sydney (UTS), Australia（悉尼大学工程与信息技术学院（UTS），澳大利亚）；School of Artificial Intelligence and Computer Science, Nantong University, China（南通大学人工智能与计算机科学学院，中国）；College of Computing and Data Science, Nanyang Technological University, Singapore（南洋理工大学计算与数据科学学院，新加坡）
Nanjing University（南京大学）；Hikvision Research Institute（海康威视研究院）
Novel classes frequently arise in our dynamically changing world, e.g., new users in the authentication system, and a machine learning model should recognize new classes without forgetting old ones. This scenario becomes more challenging when new class instances are insufficient, which is called few-shot class-incremental learning (FSCIL). Current methods handle incremental learning retrospectively by making the updated model similar to the old one. By contrast, we suggest learning prospectively to prepare for future updates, and propose ForwArd Compatible Training (FACT) for FSCIL. Forward co…
Peking University（北京大学）；University of Oxford（牛津大学）；JD Explore Academy（京东探索研究院）；Wuhan University（武汉大学）；Peng Cheng Laboratory（鹏城实验室）
Few-shot class-incremental learning (FSCIL) has been a challenging problem as only a few training samples are accessible for each novel class in the new sessions. Finetuning the backbone or adjusting the classifier prototypes trained in the prior sessions would inevitably cause a misalignment between the feature and classifier of old classes, which explains the well-known catastrophic forgetting problem. In this paper, we deal with this misalignment dilemma in FSCIL inspired by the recently discovered phenomenon named neural collapse, which reveals that the last-layer features of the same clas…
University of Toronto（多伦多大学）；Vector Institute（向量研究所）；Uber ATG（优步先进技术集团）
Machine learning classifiers are often trained to recognize a set of pre-defined classes. However, in many applications, it is often desirable to have the flexibility of learning additional concepts, with limited data and without re-training on the full training set. This paper addresses this problem, incremental few-shot learning, where a regular classification network has already been trained to recognize a set of base classes, and several extra novel classes are being considered, each with only a few labeled examples. After learning the novel classes, the model is then evaluated on the over…
Standard few-shot experiments involve learning to efficiently match previously unseen samples by class. We claim that few-shot learning should be long term, assimilating knowledge for the future, without forgetting previous concepts. In the mammalian brain, the hippocampus is understood to play a significant role in this process, by learning rapidly and consolidating knowledge to the neocortex incrementally over a short period. In this research we tested whether an artificial hippocampal algorithm (AHA), could be used with a conventional Machine Learning (ML) model that learns incrementally an…
Automatic Attack Discovery for Few-Shot Class-Incremental Learning via Large Language Models
School of Computer and Communication Engineering（计算机与通信工程学院）；School of Information and Communication（信息与通信学院）；University of Electronic Science and Technology of China（电子科学与技术大学）；Department of Strategic and Advanced Interdisciplinary Research（战略与先进跨学科研究部）；Pengcheng Laboratory（鹏城实验室）
School of Computer Science and Technology, Harbin Institute of Technology (Shenzhen)（计算机科学与技术学院，哈尔滨工业大学（深圳））；Department of Computer Science and Technology, Ocean University of China（计算机科学与技术系，中国海洋大学）；Huawei Noah’s Ark Lab（华为诺亚实验室）
SCOPE: Scene-Contextualized Incremental Few-Shot 3D Segmentation
University of Surrey, UK（英国萨里大学）；University of Wollongong, Australia（澳大利亚沃拉彭大学）；University of Copenhagen, Denmark（丹麦哥本哈根大学）；University of Exeter, UK（英国埃克塞特大学）；Singapore University of Technology and Design, Singapore（新加坡科技与设计大学）
Consistency-Driven Calibration and Matching for Few-Shot Class-Incremental Learning
School of Automation, Central South University, China（中南大学自动化学院）；Big Data Institute, Central South University, China（中南大学大数据研究院）；School of Computer Science, Faculty of Engineering, The University of Sydney, Australia（悉尼大学工程学院计算机科学系）
IPEC: Test-Time Incremental Prototype Enhancement Classifier for Few-Shot Learning
College of Intelligent Robotics and Advanced Manufacturing, Fudan University（智能机器人与先进制造学院，复旦大学）；School of Microelectronics, Fudan University（微电子学院，复旦大学）；Key Laboratory of Advanced Control and Optimization for Chemical Processes of Ministry of Education, East China University of Science and Technology（教育部先进化工过程控制与优化重点实验室，东华大学）
On Distilling the Displacement Knowledge for Few-Shot Class-Incremental Learning
School of Computer Science and Engineering, Southeast University（东南大学计算机科学与工程学院）；Key Laboratory of New Generation Artificial Intelligence Technology and Its Interdisciplinary Applications (Southeast University), Ministry of Education（教育部新一代人工智能技术及交叉应用重点实验室（东南大学））
Brain-inspired analogical mixture prototypes for few-shot class-incremental learning
Institute of Automation, Chinese Academy of Sciences（中国科学院自动化研究所）；University of Chinese Academy of Sciences（中国科学院大学）；Hong Kong Institute of Science and Innovation, Chinese Academy of Sciences（中国科学院香港创新研究院）
