We introduce a neural network architecture that logarithmically reduces the number of self-rehearsal steps in the generative rehearsal of continually learned models. In continual learning (CL), training samples come in subsequent tasks, and the trained model can access only a single task at a time. To replay previous samples, contemporary CL methods bootstrap generative models and train them recursively with a combination of current and regenerated past data. This recurrence leads to superfluous computations as the same past samples are regenerated after each task, and the reconstruction quality successively degrades. In this work, we address these limitations and propose a new generative rehearsal architecture that requires at most logarithmic number of retraining for each sample. Our approach leverages allocation of past data in a~set of generative models such that most of them do not require retraining after a~task. The experimental evaluation of our logarithmic continual learning approach shows the superiority of our method with respect to the state-of-the-art generative rehearsal methods.
Understanding the Role of Rehearsal Scale in Continual Learning under Varying Model Capacities
Institute of Intelligent Information Processing, Shanxi University（山西大学智能信息处理研究院）；Alliance Manchester Business School, The University of Manchester（曼彻斯特大学阿联酋商业学院）
The goal of continual learning (CL) is to efficiently update a machine learning model with new data without forgetting previously-learned knowledge. Most widely-used CL methods rely on a rehearsal memory of data points to be reused while training on new data. Curating such a rehearsal memory to maintain a small, informative subset of all the data seen so far is crucial to the success of these methods. We devise a coreset selection method for rehearsal-based continual learning. Our method is based on the idea of gradient matching: The gradients induced by the coreset should match, as closely as…
Suzhou University of Science and Technology（苏州科技大学）；Tianjin University（天津大学）
Rehearsal, retraining on a stored small data subset of old tasks, has been proven effective in solving catastrophic forgetting in continual learning. However, due to the sampled data may have a large bias towards the original dataset, retraining them is susceptible to driving continual domain drift of old tasks in feature space, resulting in forgetting. In this paper, we focus on tackling the continual domain drift problem with centroid distance distillation. First, we propose a centroid caching mechanism for sampling data points based on constructed centroids to reduce the sample bias in rehe…
Rehearsal-Free Modular and Compositional Continual Learning for Language Models
Bosch Center for Artificial Intelligence（博世人工智能中心）；LMU Munich（慕尼黑大学）；Hochschule der Medien（斯图加特媒体大学）；Karlsruhe University of Applied Sciences（卡尔斯鲁厄应用技术大学）
Learning from non-stationary data streams and overcoming catastrophic forgetting still poses a serious challenge for machine learning research. Rather than aiming to improve state-of-the-art, in this work we provide insight into the limits and merits of rehearsal, one of continual learning's most established methods. We hypothesize that models trained sequentially with rehearsal tend to stay in the same low-loss region after a task has finished, but are at risk of overfitting on its sample memory, hence harming generalization. We provide both conceptual and strong empirical evidence on three b…
University of Waikato（怀卡托大学）；LTCI, Télécom Paris（巴黎电信学院LTCI实验室）
Online continual learning (OCL) aims to train neural networks incrementally from a non-stationary data stream with a single pass through data. Rehearsal-based methods attempt to approximate the observed input distributions over time with a small memory and revisit them later to avoid forgetting. Despite its strong empirical performance, rehearsal methods still suffer from a poor approximation of the loss landscape of past data with memory samples. This paper revisits the rehearsal dynamics in online settings. We provide theoretical insights on the inherent memory overfitting risk from the view…
Continual learning aims to enable a single model to learn a sequence of tasks without catastrophic forgetting. Top-performing methods usually require a rehearsal buffer to store past pristine examples for experience replay, which, however, limits their practical value due to privacy and memory constraints. In this work, we present a simple yet effective framework, DualPrompt, which learns a tiny set of parameters, called prompts, to properly instruct a pre-trained model to learn tasks arriving sequentially without buffering past examples. DualPrompt presents a novel approach to attach compleme…
University of Modena and Reggio Emilia（摩德纳和雷焦艾米利亚大学）；University of Catania（卡塔尼亚大学）
Rehearsal approaches enjoy immense popularity with Continual Learning (CL) practitioners. These methods collect samples from previously encountered data distributions in a small memory buffer; subsequently, they repeatedly optimize on the latter to prevent catastrophic forgetting. This work draws attention to a hidden pitfall of this widespread practice: repeated optimization on a small pool of data inevitably leads to tight and unstable decision boundaries, which are a major hindrance to generalization. To address this issue, we propose Lipschitz-DrivEn Rehearsal (LiDER), a surrogate objectiv…
Escaping Stability-Plasticity Dilemma in Online Continual Learning for Motion Forecasting via Synergetic Memory Rehearsal
School of Mechanical Engineering, Beijing Institute of Technology（北京理工大学机械工程学院）；Beijing Institute of Technology（北京理工大学）；Zhengzhou Research Institute, Beijing Institute of Technology（北京理工大学郑州研究所）；School of Mechanical and Aerospace Engineering, Nanyang Technological University（南洋理工大学机械与航空航天工程学院）；Key Laboratory of Road and Traffic Engineering, Ministry of Education, Tongji University（同济大学交通工程重点实验室）；Zhengzhou Nissan Automobile Co., Ltd.（郑州日产汽车有限公司）
HRL Laboratories（HRL实验室）；Information and Systems Sciences Laboratory（信息与系统科学实验室）；Center for Human Machine Collaboration（人机协作中心）
The utility of learning a dynamics/world model of the environment in reinforcement learning has been shown in a many ways. When using neural networks, however, these models suffer catastrophic forgetting when learned in a lifelong or continual fashion. Current solutions to the continual learning problem require experience to be segmented and labeled as discrete tasks, however, in continuous experience it is generally unclear what a sufficient segmentation of tasks would be. Here we propose a method to continually learn these internal world models through the interleaving of internally generate…
A dataset is a shred of crucial evidence to describe a task. However, each data point in the dataset does not have the same potential, as some of the data points can be more representative or informative than others. This unequal importance among the data points may have a large impact in rehearsal-based continual learning, where we store a subset of the training examples (coreset) to be replayed later to alleviate catastrophic forgetting. In continual learning, the quality of the samples stored in the coreset directly affects the model's effectiveness and efficiency. The coreset selection pro…
Georgia Institute of Technology（佐治亚理工学院）；MIT-IBM Watson AI Lab（MIT-IBM沃森人工智能实验室）；Rice University（莱斯大学）；IBM Research（IBM研究院）
Computer vision models suffer from a phenomenon known as catastrophic forgetting when learning novel concepts from continuously shifting training data. Typical solutions for this continual learning problem require extensive rehearsal of previously seen data, which increases memory costs and may violate data privacy. Recently, the emergence of large-scale pre-trained vision transformer models has enabled prompting approaches as an alternative to data-rehearsal. These approaches rely on a key-query mechanism to generate prompts and have been found to be highly resistant to catastrophic forgettin…
Robotic vision is a field where continual learning can play a significant role. An embodied agent operating in a complex environment subject to frequent and unpredictable changes is required to learn and adapt continuously. In the context of object recognition, for example, a robot should be able to learn (without forgetting) objects of never before seen classes as well as improving its recognition capabilities as new instances of already known classes are discovered. Ideally, continual learning should be triggered by the availability of short videos of single objects and performed on-line on …
University of Wroclaw（弗罗茨瓦夫大学）；Warsaw University of Technology（华沙工业大学）；Tooploox
Contemporary deep neural networks offer state-of-the-art results when applied to visual reasoning, e.g., in the context of 3D point cloud data. Point clouds are important datatype for precise modeling of three-dimensional environments, but effective processing of this type of data proves to be challenging. In the world of large, heavily-parameterized network architectures and continuously-streamed data, there is an increasing need for machine learning models that can be trained on additional data. Unfortunately, currently available models cannot fully leverage training on additional data witho…
Overcoming Domain Drift in Online Continual Learning
Institute of Automation, Chinese Academy of Sciences（中国科学院自动化研究所）；Suzhou University of Science and Technology（苏州科技大学）；Youtu Lab, Tencent（腾讯优图实验室）；College of Intelligence and Computing, Tianjin University（天津大学智能与计算学部）
We devise a coreset selection method based on the idea of gradient matching: The gradients induced by the coreset should match, as closely as possible, those induced by the original training dataset. We evaluate the method in the context of continual learning, where it can be used to curate a rehearsal memory. Our method performs strong competitors such as reservoir sampling across a range of memory sizes.
Continual Learning from Simulated Interactions via Multitask Prospective Rehearsal for Bionic Limb Behavior Modeling
ETH Zurich（苏黎世联邦理工学院）；University of Goettingen（戈丁根大学）；Bielefeld University（比勒菲尔德大学）；European Neuroscience Institute, Goettingen (ENI-G)（戈丁根欧洲神经科学研究所）；Qatar Computing Institute（卡塔尔计算研究所）；University Medical Center, Goettingen (UMG)（戈丁根大学医学中心）
DESIRE: Dynamic Knowledge Consolidation for Rehearsal-Free Continual Learning
Institute of Automation, Chinese Academy of Sciences（中国科学院自动化研究所）；School of Artificial Intelligence, University of Chinese Academy of Sciences（中国科学院大学人工智能学院）；Centre for Artificial Intelligence and Robotics, HKISI-CAS（香港中文大学（深圳）人工智能与机器人中心（中国科学院深圳先进技术研究院））；Department of Computer Science, University of Illinois at Chicago（芝加哥大学计算机科学系）
Consistent Prompting for Rehearsal-Free Continual Learning
School of Artificial Intelligence, Sun Yat-sen University（中山大学人工智能学院）；Cheng Kar-Shun Robotics Institute, The Hong Kong University of Science and Technology（香港科技大学郑家纯机器人研究院）；Key Laboratory of Machine Intelligence and Advanced Computing, Ministry of Education（教育部机器智能与先进计算重点实验室）
The mainstream paradigm behind continual learning has been to adapt the model parameters to non-stationary data distributions, where catastrophic forgetting is the central challenge. Typical methods rely on a rehearsal buffer or known task identity at test time to retrieve learned knowledge and address forgetting, while this work presents a new paradigm for continual learning that aims to train a more succinct memory system without accessing task identity at test time. Our method learns to dynamically prompt (L2P) a pre-trained model to learn tasks sequentially under different task transitions…
Combining Statistical Features and Deep Encodings for Rehearsal-Based Class-Incremental Time Series Classification
atlanTTic – ICLAB, Universidade de Vigo（atlanTTic–ICLAB，维戈大学）；Centro Tecnolóxico de Telecomunicacións de Galicia (GRADIANT)（加利西亚电信技术中心（GRADIANT））；Universidade de Vigo（维戈大学）
PROL : Rehearsal Free Continual Learning in Streaming Data via Prompt Online Learning
STEM University of South Australia（南澳大利亚STEM大学）；Institute for Infocomm Research, A*STAR & IPAL（信息与通信研究所，A*STAR及IPAL）；Systems Research Institute, Polish Academy of Sciences（波兰科学院系统研究所）
Eindhoven University of Technology（埃因霍温理工大学）；University of Twente（特文特大学）
Continual learning aims to provide intelligent agents that are capable of learning continually a sequence of tasks, building on previously learned knowledge. A key challenge in this learning paradigm is catastrophically forgetting previously learned tasks when the agent faces a new one. Current rehearsal-based methods show their success in mitigating the catastrophic forgetting problem by replaying samples from previous tasks during learning a new one. However, these methods are infeasible when the data of previous tasks is not accessible. In this work, we propose a new pseudo-rehearsal-based …
Warsaw University of Technology（华沙理工大学）；Jagiellonian University（雅盖隆大学）；Tooploox
We introduce a new method for internal replay that modulates the frequency of rehearsal based on the depth of the network. While replay strategies mitigate the effects of catastrophic forgetting in neural networks, recent works on generative replay show that performing the rehearsal only on the deeper layers of the network improves the performance in continual learning. However, the generative approach introduces additional computational overhead, limiting its applications. Motivated by the observation that earlier layers of neural networks forget less abruptly, we propose to update network la…
Institute of Signal Processing and System Theory（信号处理与系统理论研究所）；University of Stuttgart（斯图加特大学）
Deep Neural Networks (DNNs) suffer from a rapid decrease in performance when trained on a sequence of tasks where only data of the most recent task is available. This phenomenon, known as catastrophic forgetting, prevents DNNs from accumulating knowledge over time. Overcoming catastrophic forgetting and enabling continual learning is of great interest since it would enable the application of DNNs in settings where unrestricted access to all the training data at any time is not always possible, e.g. due to storage limitations or legal issues. While many recently proposed methods for continual l…
University of Modena and Reggio Emilia（摩德纳-雷焦艾米利亚大学）
In Continual Learning, a Neural Network is trained on a stream of data whose distribution shifts over time. Under these assumptions, it is especially challenging to improve on classes appearing later in the stream while remaining accurate on previous ones. This is due to the infamous problem of catastrophic forgetting, which causes a quick performance degradation when the classifier focuses on learning new categories. Recent literature proposed various approaches to tackle this issue, often resorting to very sophisticated techniques. In this work, we show that naive rehearsal can be patched to…
NavInfo Europe（四维图新欧洲）；Eindhoven University of Technology（埃因霍温理工大学）
Real-world applications often require learning continuously from a stream of data under ever-changing conditions. When trying to learn from such non-stationary data, deep neural networks (DNNs) undergo catastrophic forgetting of previously learned information. Among the common approaches to avoid catastrophic forgetting, rehearsal-based methods have proven effective. However, they are still prone to forgetting due to task-interference as all parameters respond to all tasks. To counter this, we take inspiration from sparse coding in the brain and introduce dynamic modularity and sparsity (Dynam…
