Using backward error analysis, we compute implicit training biases in multitask and continual learning settings for neural networks trained with stochastic gradient descent. In particular, we derive modified losses that are implicitly minimized during training. They have three terms: the original loss, accounting for convergence, an implicit flatness regularization term proportional to the learning rate, and a last term, the conflict term, which can theoretically be detrimental to both convergence and implicit regularization. In multitask, the conflict term is a well-known quantity, measuring the gradient alignment between the tasks, while in continual learning the conflict term is a new quantity in deep learning optimization, although a basic tool in differential geometry: The Lie bracket between the task gradients.
Washington State University（华盛顿州立大学）；DeepMind（深度思维）
Continual (sequential) training and multitask (simultaneous) training are often attempting to solve the same overall objective: to find a solution that performs well on all considered tasks. The main difference is in the training regimes, where continual learning can only have access to one task at a time, which for neural networks typically leads to catastrophic forgetting. That is, the solution found for a subsequent task does not perform well on the previous ones anymore. However, the relationship between the different minima that the two training regimes arrive at is not well understood. W…
Continual Learning from Simulated Interactions via Multitask Prospective Rehearsal for Bionic Limb Behavior Modeling
ETH Zurich（苏黎世联邦理工学院）；University of Goettingen（戈丁根大学）；Bielefeld University（比勒菲尔德大学）；European Neuroscience Institute, Goettingen (ENI-G)（戈丁根欧洲神经科学研究所）；Qatar Computing Institute（卡塔尔计算研究所）；University Medical Center, Goettingen (UMG)（戈丁根大学医学中心）
Arizona State University（亚利桑那州立大学）；University of California, Davis（加利福尼亚大学戴维斯分校）
By learning a sequence of tasks continually, an agent in continual learning (CL) can improve the learning performance of both a new task and `old' tasks by leveraging the forward knowledge transfer and the backward knowledge transfer, respectively. However, most existing CL methods focus on addressing catastrophic forgetting in neural networks by minimizing the modification of the learnt model for old tasks. This inevitably limits the backward knowledge transfer from the new task to the old tasks, because judicious model updates could possibly improve the learning performance of the old tasks …
Continual learning of a stream of tasks is an active area in deep neural networks. The main challenge investigated has been the phenomenon of catastrophic forgetting or interference of newly acquired knowledge with knowledge from previous tasks. Recent work has investigated forward knowledge transfer to new tasks. Backward transfer for improving knowledge gained during previous tasks has received much less attention. There is in general limited understanding of how knowledge transfer could aid tasks learned continually. We present a theory for knowledge transfer in continual supervised learnin…
Towards stable training of parallel continual learning
College of Intelligence and Computing, Tianjin University（天津大学智能与计算学部）；Institute of Automation, Chinese Academy of Sciences（中国科学院自动化研究所）；School of Automation, Southeast University（东南大学自动化学院）
Multitask learning assumes that models capable of learning from multiple tasks can achieve better quality and efficiency via knowledge transfer, a key feature of human learning. Though, state of the art ML models rely on high customization for each task and leverage size and data scale rather than scaling the number of tasks. Also, continual learning, that adds the temporal aspect to multitask, is often focused to the study of common pitfalls such as catastrophic forgetting instead of being studied at a large scale as a critical component to build the next generation artificial intelligence.We…
Quantum continual learning of quantum data realizing knowledge backward transfer
South China Agricultural University（华南农业大学）；Sun Yat-sen University（中山大学）；Guilin University of Electronic Technology（桂林电子科技大学）；Tsinghua University（清华大学）
University of Cambridge（剑桥大学）；University of Sydney（悉尼大学）
In the continual learning setting, tasks are encountered sequentially. The goal is to learn whilst i) avoiding catastrophic forgetting, ii) efficiently using model capacity, and iii) employing forward and backward transfer learning. In this paper, we explore how the Variational Continual Learning (VCL) framework achieves these desiderata on two benchmarks in continual learning: split MNIST and permuted MNIST. We first report significantly improved results on what was already a competitive approach. The improvements are achieved by establishing a new best practice approach to mean-field variati…
GRID: Scaling Task-Agnostic Inference in Continual Prompt Tuning
State University of New York at Buffalo（纽约州立大学布法罗分校）；Department of Computer Science and Engineering（计算机科学与工程系）；Institute for Artificial Intelligence and Data Science（人工智能与数据科学研究院）
Indian Institute of Technology (BHU), Varanasi（印度理工学院（巴纳拉斯印度教大学）瓦拉纳西分校）；Massachusetts Institute of Technology（麻省理工学院）
The Continual Learning (CL) problem involves performing well on a sequence of tasks under limited compute. Current algorithms in the domain are either slow, offline or sensitive to hyper-parameters. La-MAML, an optimization-based meta-learning algorithm claims to be better than other replay-based, prior-based and meta-learning based approaches. According to the MER paper [1], metrics to measure performance in the continual learning arena are Retained Accuracy (RA) and Backward Transfer-Interference (BTI). La-MAML claims to perform better in these values when compared to the SOTA in the domain.…
Interleaved Multitask Learning with Energy Modulated Learning Progress
Artificial Intelligence and Data Engineering Department, Ozyegin University（奥克琴大学人工智能与数据工程系）；Department of Computer Engineering, Bogazici University（博雅奇大学计算机工程系）；SISREC, OTRI, University of Osaka（大阪大学SISREC、OTRI）；International Professional University of Technology（国际专业技术大学）
ENSTA ParisTech（巴黎高等先进技术学院）；INRIA Flowers（法国国家信息与自动化研究所Flowers实验室）；University of Bologna（博洛尼亚大学）
Continual learning consists of algorithms that learn from a stream of data/tasks continuously and adaptively thought time, enabling the incremental development of ever more complex knowledge and skills. The lack of consensus in evaluating continual learning algorithms and the almost exclusive focus on forgetting motivate us to propose a more comprehensive set of implementation independent metrics accounting for several factors we believe have practical implications worth considering in the deployment of real AI systems that learn continually: accuracy or performance over time, backward and for…
RISE Research Institutes of Sweden（瑞典RISE研究所）；KTH Royal Institute of Technology（瑞典皇家理工学院）；Université de Montréal（蒙特利尔大学）；MILA-Quebec AI Institute（魁北克MILA人工智能研究所）；Karolinska Institutet（卡罗林斯卡学院）；MedTechLabs（医疗技术实验室）；Uppsala University（乌普萨拉大学）
Baylor College of Medicine（贝勒医学院）；Rice University（莱斯大学）
A major obstacle to developing artificial intelligence applications capable of true lifelong learning is that artificial neural networks quickly or catastrophically forget previously learned tasks when trained on a new one. Numerous methods for alleviating catastrophic forgetting are currently being proposed, but differences in evaluation protocols make it difficult to directly compare their performance. To enable more meaningful comparisons, here we identified three distinct scenarios for continual learning based on whether task identity is known and, if it is not, whether it needs to be infe…
Modular Delta Merging with Orthogonal Constraints: A Scalable Framework for Continual and Reversible Model Composition
Rensselaer Polytechnic Institute（伦斯勒理工学院）；National University of Sciences and Technology (NUST)（国立科技大学）；Virginia Tech（弗吉尼亚理工大学）；University of New Mexico（新墨西哥大学）
University of Vermont（佛蒙特大学）；Uber AI Labs（优步人工智能实验室）；OpenAI（开放人工智能研究中心）
Continual lifelong learning requires an agent or model to learn many sequentially ordered tasks, building on previous knowledge without catastrophically forgetting it. Much work has gone towards preventing the default tendency of machine learning models to catastrophically forget, yet virtually all such work involves manually-designed solutions to the problem. We instead advocate meta-learning a solution to catastrophic forgetting, allowing AI to learn to continually learn. Inspired by neuromodulatory processes in the brain, we propose A Neuromodulated Meta-Learning Algorithm (ANML). It differ…
University of Illinois at Chicago（伊利诺伊大学芝加哥分校）；Southwest Jiaotong University（西南交通大学）
This paper studies continual learning (CL) for sentiment classification (SC). In this setting, the CL system learns a sequence of SC tasks incrementally in a neural network, where each task builds a classifier to classify the sentiment of reviews of a particular product category or domain. Two natural questions are: Can the system transfer the knowledge learned in the past from the previous tasks to the new task to help it learn a better model for the new task? And, can old models for previous tasks be improved in the process as well? This paper proposes a novel technique called KAN to achieve…
Sungkyunkwan University（成均馆大学）；Seoul National University（首尔国立大学）
Most continual learning (CL) algorithms have focused on tackling the stability-plasticity dilemma, that is, the challenge of preventing the forgetting of previous tasks while learning new ones. However, they have overlooked the impact of the knowledge transfer when the dataset in a certain task is biased - namely, when some unintended spurious correlations of the tasks are learned from the biased dataset. In that case, how would they affect learning future tasks or the knowledge already learned from the past tasks? In this work, we carefully design systematic experiments using one synthetic an…
Minimax Forward and Backward Learning of Evolving Tasks with Performance Guarantees
Basque Center of Applied Mathematics (BCAM)（巴斯克应用数学中心）；IKERBASQUE-Basque Foundation for Science（伊克尔巴斯克科学基金会）；University of the Basque Country UPV/EHU（巴斯克大学）
University of Illinois at Chicago（伊利诺伊大学芝加哥分校）；Facebook AI Research（Facebook人工智能研究院）
This paper studies continual learning (CL) of a sequence of aspect sentiment classification (ASC) tasks. Although some CL techniques have been proposed for document sentiment classification, we are not aware of any CL work on ASC. A CL system that incrementally learns a sequence of ASC tasks should address the following two issues: (1) transfer knowledge learned from previous tasks to the new task to help it learn a better model, and (2) maintain the performance of the models for previous tasks so that they are not forgotten. This paper proposes a novel capsule network based model called B-CL …
Merge then Realign: Simple and Effective Modality-Incremental Continual Learning for Multimodal LLMs
Harbin Institute of Technology, Shenzhen（哈尔滨工业大学（深圳））；Guangdong Provincial Key Laboratory of Novel Security Intelligence Technologies（广东省新型安全智能技术重点实验室）
G2LoRA: Gradient Orthogonal Low-Rank Adaptation Framework for Graph Continual Learning on Text-Attributed Graphs
School of Computer Science and Engineering, Beihang University（北航计算机科学与工程学院）；Department of Statistics, Columbia University（哥伦比亚大学统计系）；College of Computer Science, Beijing University of Technology（北京理工大学计算机学院）
University of Illinois at Chicago（伊利诺伊大学芝加哥分校）；Peking University（北京大学）；ETH Zurich（苏黎世联邦理工学院）
Existing research on continual learning of a sequence of tasks focused on dealing with catastrophic forgetting, where the tasks are assumed to be dissimilar and have little shared knowledge. Some work has also been done to transfer previously learned knowledge to the new task when the tasks are similar and have shared knowledge. To the best of our knowledge, no technique has been proposed to learn a sequence of mixed similar and dissimilar tasks that can deal with forgetting and also transfer knowledge forward and backward. This paper proposes such a technique to learn both types of tasks in t…
Warsaw University of Technology（华沙理工大学）；Jagiellonian University（雅盖隆大学）；Tooploox
We propose a new method for unsupervised generative continual learning through realignment of Variational Autoencoder's latent space. Deep generative models suffer from catastrophic forgetting in the same way as other neural structures. Recent generative continual learning works approach this problem and try to learn from new data without forgetting previous knowledge. However, those methods usually focus on artificial scenarios where examples share almost no similarity between subsequent portions of data - an assumption not realistic in the real-life applications of continual learning. In thi…
Continual learning is considered a promising step towards next-generation Artificial Intelligence (AI), where deep neural networks (DNNs) make decisions by continuously learning a sequence of different tasks akin to human learning processes. It is still quite primitive, with existing works focusing primarily on avoiding (catastrophic) forgetting. However, since forgetting is inevitable given bounded memory and unbounded task loads, 'how to reasonably forget' is a problem continual learning must address in order to reduce the performance gap between AIs and humans, in terms of 1) memory efficie…
SAE-FD: Sparse Autoencoder Feature Distillation for Continual Learning of Large Language Models
The Hong Kong University of Science and Technology (Guangzhou)（香港科技大学（广州））；Nanjing University of Aeronautics and Astronautics（南京航空航天大学）；The 63rd Research Institute, National University of Defense Technology, Nanjing（国防科技大学第六十三研究所，南京）
ServiceNow；Mila - Quebec AI Institute（米拉-魁北克人工智能研究所）；Université de Montréal（蒙特利尔大学）；HEC Montréal（蒙特利尔高等商学院）
Modularity is a compelling solution to continual learning (CL), the problem of modeling sequences of related tasks. Learning and then composing modules to solve different tasks provides an abstraction to address the principal challenges of CL including catastrophic forgetting, backward and forward transfer across tasks, and sub-linear model growth. We introduce local module composition (LMC), an approach to modular CL where each module is provided a local structural component that estimates a module's relevance to the input. Dynamic module composition is performed layer-wise based on local rel…
University of Bologna（博洛尼亚大学）；University of Modena and Reggio Emilia（摩德纳-雷焦艾米利亚大学）；ETH Zürich（苏黎世联邦理工学院）
In the last few years, research and development on Deep Learning models and techniques for ultra-low-power devices in a word, TinyML has mainly focused on a train-then-deploy assumption, with static models that cannot be adapted to newly collected data without cloud-based data collection and fine-tuning. Latent Replay-based Continual Learning (CL) techniques[1] enable online, serverless adaptation in principle, but so farthey have still been too computation and memory-hungry for ultra-low-power TinyML devices, which are typically based on microcontrollers. In this work, we introduce a HW/SW pl…
Polytechnique Montreal（蒙特利尔理工学院）；MILA, Quebec AI Institute（魁北克人工智能研究所MILA）；CHU Rennes（雷恩大学中心医院）；CHU Montpellier（蒙彼利埃大学中心医院）；Karolinska Institutet（卡罗林斯卡学院）；University of California, San Francisco（加州大学旧金山分校）；National Institute of Neurological Disorders and Stroke, National Institutes of Health（美国国立卫生研究院国家神经疾病与中风研究所）；IRCCS San Raffaele Scientific Institute（IRCCS圣拉斐尔科学研究所）；Vita-Salute San Raffaele University（圣拉斐尔生命健康大学）；Brigham and Women’s Hospital（布莱根妇女医院）；Harvard Medical School（哈佛医学院）；AP-HM, CHU Timone（马赛公共医院集团蒂莫内大学中心医院）；Aix-Marseille Univ（艾克斯-马赛大学）；CNRS（法国国家科学研究中心）
Segmentation of Multiple Sclerosis (MS) lesions is a challenging problem. Several deep-learning-based methods have been proposed in recent years. However, most methods tend to be static, that is, a single model trained on a large, specialized dataset, which does not generalize well. Instead, the model should learn across datasets arriving sequentially from different hospitals by building upon the characteristics of lesions in a continual manner. In this regard, we explore experience replay, a well-known continual learning method, in the context of MS lesion segmentation across multi-contrast d…
