Incremental learning attempts to develop a classifier which learns continuously from a stream of data segregated into different classes. Deep learning approaches suffer from catastrophic forgetting when learning classes incrementally, while most incremental learning approaches require a large amount of training data per class. We examine the problem of incremental learning using only a few training examples, referred to as Few-Shot Incremental Learning (FSIL). To solve this problem, we propose a novel approach inspired by the concept learning model of the hippocampus and the neocortex that represents each image class as centroids and does not suffer from catastrophic forgetting. We evaluate our approach on three class-incremental learning benchmarks: Caltech-101, CUBS-200-2011 and CIFAR-100 for incremental and few-shot incremental learning and show that our approach achieves state-of-the-art results in terms of classification accuracy over all learned classes.
NumerosityVLM: A Cognitively Inspired Benchmark for Interpreting Numerosity Representations in Vision-Language Models
The University of Manchester（曼彻斯特大学）；University of Michigan（密歇根大学）；Sheffield Hallam University（谢菲尔德哈勒姆大学）；Tufts University（塔夫茨大学）；ENSTA, Institut Polytechnique de Paris（巴黎综合理工学院国立高等先进技术学校）
This paper proposes the CogSense system, which is inspired by sense-making cognition and perception in the mammalian brain to perform perception error detection and perception parameter adaptation using probabilistic signal temporal logic. As a specific application, a contrast-based perception adaption method is presented and validated. The proposed method evaluates perception errors using heterogeneous probe functions computed from the detected objects and subsequently solves a contrast optimization problem to correct perception errors. The CogSense probe functions utilize the characteristics…
Rochester Institute of Technology（罗切斯特理工学院）；Carlson Center for Imaging Science（卡尔森成像科学中心）
Incremental class learning involves sequentially learning classes in bursts of examples from the same class. This violates the assumptions that underlie methods for training standard deep neural networks, and will cause them to suffer from catastrophic forgetting. Arguably, the best method for incremental class learning is iCaRL, but it requires storing training examples for each class, making it challenging to scale. Here, we propose FearNet for incremental class learning. FearNet is a generative model that does not store previous examples, making it memory efficient. FearNet uses a brain-ins…
Symphony: A Cognitively-Inspired Multi-Agent System for Long-Video Understanding
Institute of Automation, Chinese Academy of Sciences（中国科学院自动化研究所）；Kuaishou Technology（快手科技）；School of Future Technology, University of Chinese Academy of Sciences（中国科学院大学未来技术学院）
CogRad: A Cognitively-Inspired Multi-Agent Framework for Radiology Report Generation
German Research Center for Artificial Intelligence（德国人工智能研究中心）；Intelligentx GmbH（智能X有限公司）；Department of Core Informatics, Graduate School of Informatics, Osaka Metropolitan University（大阪城市大学信息科学研究生院核心信息学系）
Purdue University（普渡大学）；JD AI Research（京东人工智能研究院）；University of Rochester（罗切斯特大学）
Some cognitive research has discovered that humans accomplish event segmentation as a side effect of event anticipation. Inspired by this discovery, we propose a simple yet effective end-to-end self-supervised learning framework for event segmentation/boundary detection. Unlike the mainstream clustering-based methods, our framework exploits a transformer-based feature reconstruction scheme to detect event boundary by reconstruction errors. This is consistent with the fact that humans spot new events by leveraging the deviation between their prediction and what is actually perceived. Thanks to …
Peking University（北京大学）；University of Oxford（牛津大学）；JD Explore Academy（京东探索研究院）；Wuhan University（武汉大学）；Peng Cheng Laboratory（鹏城实验室）
Few-shot class-incremental learning (FSCIL) has been a challenging problem as only a few training samples are accessible for each novel class in the new sessions. Finetuning the backbone or adjusting the classifier prototypes trained in the prior sessions would inevitably cause a misalignment between the feature and classifier of old classes, which explains the well-known catastrophic forgetting problem. In this paper, we deal with this misalignment dilemma in FSCIL inspired by the recently discovered phenomenon named neural collapse, which reveals that the last-layer features of the same clas…
Brain-inspired analogical mixture prototypes for few-shot class-incremental learning
Institute of Automation, Chinese Academy of Sciences（中国科学院自动化研究所）；University of Chinese Academy of Sciences（中国科学院大学）；Hong Kong Institute of Science and Innovation, Chinese Academy of Sciences（中国科学院香港创新研究院）
This paper contributes a novel cognitively-inspired method for RGB-D indoor scene classification. High intra-class variance and low inter-class variance make indoor scene classification an extremely challenging task. To cope with this problem, we propose a clustering approach inspired by the concept learning model of the hippocampus and the neocortex, to generate clusters and centroids for different scene categories. Test images depicting different scenes are classified by using their distance to the closest centroids (concepts). Modeling of RGB-D scenes as centroids not only leads to state-of…
Class-incremental learning (CIL) learns a classification model with training data of different classes arising progressively. Existing CIL either suffers from serious accuracy loss due to catastrophic forgetting, or invades data privacy by revisiting used exemplars. Inspired by linear learning formulations, we propose an analytic class-incremental learning (ACIL) with absolute memorization of past knowledge while avoiding breaching of data privacy (i.e., without storing historical data). The absolute memorization is demonstrated in the sense that class-incremental learning using ACIL given pre…
Learn by Reasoning: Analogical Weight Generation for Few-Shot Class-Incremental Learning
State Key Laboratory of Human-Machine Hybrid Augmented Intelligence, Institute of Artificial Intelligence and Robotics, Xi’an Jiaotong University（人机混合增强智能国家重点实验室，人工智能与机器人研究院，西安交通大学）；Faculty of Microelectronics, Shenzhen University of Advanced Technology（微电子学院，深圳先进技术大学）；College of Software Engineering, Xi’an Jiaotong University（软件工程学院，西安交通大学）
We introduce a practical Domain Adaptation (DA) paradigm called Class-Incremental Domain Adaptation (CIDA). Existing DA methods tackle domain-shift but are unsuitable for learning novel target-domain classes. Meanwhile, class-incremental (CI) methods enable learning of new classes in absence of source training data but fail under a domain-shift without labeled supervision. In this work, we effectively identify the limitations of these approaches in the CIDA paradigm. Motivated by theoretical and empirical observations, we propose an effective method, inspired by prototypical networks, that ena…
Chinese Academy of Science（中国科学院）；Harbin Institute of Technology（哈尔滨工业大学）；Peng Cheng Laboratory（鹏城实验室）；Xi’an Jiaotong University（西安交通大学）；Jinan University（暨南大学）
Although data-free incremental learning methods are memory-friendly, accurately estimating and counteracting representation shifts is challenging in the absence of historical data. This paper addresses this thorny problem by proposing a novel incremental learning method inspired by human analogy capabilities. Specifically, we design an analogy-making mechanism to remap the new data into the old class by prompt tuning. It mimics the feature distribution of the target old class on the old model using only samples of new classes. The learnt prompts are further used to estimate and counteract the …
Few-shot Class-Incremental Learning (FSCIL) aims at learning new concepts continually with only a few samples, which is prone to suffer the catastrophic forgetting and overfitting problems. The inaccessibility of old classes and the scarcity of the novel samples make it formidable to realize the trade-off between retaining old knowledge and learning novel concepts. Inspired by that different models memorize different knowledge when learning novel concepts, we propose a Memorizing Complementation Network (MCNet) to ensemble multiple models that complements the different memorized knowledge with…
CD^2: Constrained Dataset Distillation for Few-Shot Class-Incremental Learning
Institute of Information Engineering, Chinese Academy of Sciences（中国科学院信息工程研究所）；School of Cyber Security, University of Chinese Academy of Sciences（中国科学院大学网络安全学院）；Hong Kong University of Science and Technology (Guangzhou)（香港科技大学（广州））
Heuritech；Sorbonne University（巴黎索邦大学）；University of Campinas（坎皮纳斯大学）
Lifelong learning has attracted much attention, but existing works still struggle to fight catastrophic forgetting and accumulate knowledge over long stretches of incremental learning. In this work, we propose PODNet, a model inspired by representation learning. By carefully balancing the compromise between remembering the old classes and learning new ones, PODNet fights catastrophic forgetting, even over very long runs of small incremental tasks --a setting so far unexplored by current works. PODNet innovates on existing art with an efficient spatial-based distillation-loss applied throughout…
University of Windsor（温莎大学）；Higher School of Communication of Tunis（突尼斯高等通信学院）
In this paper we outline a PhD research plan. This research contributes to the field of one-class incremental learning and classification in case of non-stationary environments. The goal of this PhD is to define a new classification framework able to deal with very small learning dataset at the beginning of the process and with abilities to adjust itself according to the variability of the incoming data which create large scale datasets. As a preliminary work, incremental Covariance-guided One-Class Support Vector Machine is proposed to deal with sequentially obtained data. It is inspired from…
University of Trento（特伦托大学）；Fondazione Bruno Kessler（布鲁诺·凯塞勒基金会）
We study the new task of class-incremental Novel Class Discovery (class-iNCD), which refers to the problem of discovering novel categories in an unlabelled data set by leveraging a pre-trained model that has been trained on a labelled data set containing disjoint yet related categories. Apart from discovering novel classes, we also aim at preserving the ability of the model to recognize previously seen base categories. Inspired by rehearsal-based incremental learning methods, in this paper we propose a novel approach for class-iNCD which prevents forgetting of past information about the base c…
The two main impediments to continual learning are catastrophic forgetting and memory limitations on the storage of data. To cope with these challenges, we propose a novel, cognitively-inspired approach which trains autoencoders with Neural Style Transfer to encode and store images. During training on a new task, reconstructed images from encoded episodes are replayed in order to avoid catastrophic forgetting. The loss function for the reconstructed images is weighted to reduce its effect during classifier training to cope with image degradation. When the system runs out of memory the encoded …
The ability to learn new concepts continually is necessary in this ever-changing world. However, deep neural networks suffer from catastrophic forgetting when learning new categories. Many works have been proposed to alleviate this phenomenon, whereas most of them either fall into the stability-plasticity dilemma or take too much computation or storage overhead. Inspired by the gradient boosting algorithm to gradually fit the residuals between the target model and the previous ensemble model, we propose a novel two-stage learning paradigm FOSTER, empowering the model to learn new categories ad…
School of Computer Science and Technology, Huazhong University of Science and Technology（华中科技大学计算机科学与技术学院）；School of Computer Science, Peking University（北京大学计算机学院）
Sharif University of Technology（谢里夫理工大学）；Institute For Research In Fundamental Sciences (IPM)（基础科学研究所（IPM））
This paper presents a practical and simple yet efficient method to effectively deal with the catastrophic forgetting for Class Incremental Learning (CIL) tasks. CIL tends to learn new concepts perfectly, but not at the expense of performance and accuracy for old data. Learning new knowledge in the absence of data instances from previous classes or even imbalance samples of both old and new classes makes CIL an ongoing challenging problem. These issues can be tackled by storing exemplars belonging to the previous tasks or by utilizing the rehearsal strategy. Inspired by the rehearsal strategy w…
Rochester Institute of Technology（罗切斯特理工学院）；Adobe Research（Adobe研究院）；Paige；Cornell Tech（康奈尔科技校区）
People learn throughout life. However, incrementally updating conventional neural networks leads to catastrophic forgetting. A common remedy is replay, which is inspired by how the brain consolidates memory. Replay involves fine-tuning a network on a mixture of new and old instances. While there is neuroscientific evidence that the brain replays compressed memories, existing methods for convolutional networks replay raw images. Here, we propose REMIND, a brain-inspired approach that enables efficient replay with compressed representations. REMIND is trained in an online manner, meaning it lear…
Consistency-Driven Calibration and Matching for Few-Shot Class-Incremental Learning
School of Automation, Central South University, China（中南大学自动化学院）；Big Data Institute, Central South University, China（中南大学大数据研究院）；School of Computer Science, Faculty of Engineering, The University of Sydney, Australia（悉尼大学工程学院计算机科学系）
Optical-Guided Neural Collapse for SAR Few-Shot Class Incremental Learning
Beihang University, Beijing 100191, China（北航，北京100191，中国）；Aerospace Information Research Institute, Chinese Academy of Sciences, Beijing 100190, China（航天信息研究所，中国科学院，北京100190，中国）
Multi-View Incremental Learning with Structured Hebbian Plasticity for Enhanced Fusion Efficiency
Fuzhou University（福州大学）；Guangdong Institute of Intelligence Science and Technology（广东智能科学与技术研究院）；University of Electronic Science and Technology of China（电子科技大学）；Tsinghua University（清华大学）；Dalian University of Technology（大连理工大学）
The two main challenges faced by continual learning approaches are catastrophic forgetting and memory limitations on the storage of data. To cope with these challenges, we propose a novel, cognitively-inspired approach which trains autoencoders with Neural Style Transfer to encode and store images. Reconstructed images from encoded episodes are replayed when training the classifier model on a new task to avoid catastrophic forgetting. The loss function for the reconstructed images is weighted to reduce its effect during classifier training to cope with image degradation. When the system runs o…
The University of Queensland（昆士兰大学）；Sullivan Nicolaides Pathology（沙利文·尼科莱德斯病理中心）
The continual appearance of new objects in the visual world poses considerable challenges for current deep learning methods in real-world deployments. The challenge of new task learning is often exacerbated by the scarcity of data for the new categories due to rarity or cost. Here we explore the important task of Few-Shot Class-Incremental Learning (FSCIL) and its extreme data scarcity condition of one-shot. An ideal FSCIL model needs to perform well on all classes, regardless of their presentation order or paucity of data. It also needs to be robust to open-set real-world conditions and be ea…
Harbin Institute of Technology（哈尔滨工业大学）；University of Trento（特伦托大学）；Hong Kong University of Science and Technology（香港科技大学）；INRIA（法国国家信息与自动化研究所）；SAP AI Research（思爱普人工智能研究院）；Fondazione Bruno Kessler（布鲁诺·凯斯勒基金会）
A fundamental and challenging problem in deep learning is catastrophic forgetting, i.e. the tendency of neural networks to fail to preserve the knowledge acquired from old tasks when learning new tasks. This problem has been widely investigated in the research community and several Incremental Learning (IL) approaches have been proposed in the past years. While earlier works in computer vision have mostly focused on image classification and object detection, more recently some IL approaches for semantic segmentation have been introduced. These previous works showed that, despite its simplicity…
Human beings are able to master a variety of knowledge and skills with ongoing learning. By contrast, dramatic performance degradation is observed when new tasks are added to an existing neural network model. This phenomenon, termed as \emph{Catastrophic Forgetting}, is one of the major roadblocks that prevent deep neural networks from achieving human-level artificial intelligence. Several research efforts, e.g. \emph{Lifelong} or \emph{Continual} learning algorithms, have been proposed to tackle this problem. However, they either suffer from an accumulating drop in performance as the task seq…
Continual Learning on CLIP via Incremental Prompt Tuning with Intrinsic Textual Anchors
School of Computer Science and Engineering, University of New South Wales（新南威尔士大学计算机科学与工程学院）；Data61, CSIRO（CSIRO数据61研究中心）；School of Computer Science, University of Auckland（奥克兰大学计算机科学学院）；Australian Institute for Machine Learning (AIML), The University of Adelaide（阿德莱德大学澳大利亚机器学习研究所）
Bi-CRCL: Bidirectional Conservative-Radical Complementary Learning with Pre-trained Foundation Models for Class-incremental Medical Image Analysis
Department of Biomedical Engineering, The Chinese University of Hong Kong（生物医学工程系，香港中文大学）；Department of Radiation Oncology, Columbia University Irving Medical Center and Data Science Institute, Columbia University（放射肿瘤学系，哥伦比亚大学伊万杰琳医学中心及数据科学研究院，哥伦比亚大学）；Department of Electrical and Electronic Engineering and School of Biomedical Engineering, The University of Hong Kong（电气电子工程系和生物医学工程学院，香港大学）；Department of Computer Science, City University of Hong Kong（计算机科学系，城市大学）；Department of Artificial Intelligence, Westlake University（人工智能系，西湖大学）
