Continual Learning addresses the challenge of learning a number of different tasks sequentially. The goal of maintaining knowledge of earlier tasks without re-accessing them starkly conflicts with standard SGD training for artificial neural networks. An influential method to tackle this problem without storing old data are so-called regularisation approaches. They measure the importance of each parameter for solving a given task and subsequently protect important parameters from large changes. In the literature, three ways to measure parameter importance have been put forward and they have inspired a large body of follow-up work. Here, we present strong theoretical and empirical evidence that these three methods, Elastic Weight Consolidation (EWC), Synaptic Intelligence (SI) and Memory Aware Synapses (MAS), are surprisingly similar and are all linked to the same theoretical quantity. Concretely, we show that, despite stemming from very different motivations, both SI and MAS approximate the square root of the Fisher Information, with the Fisher being the theoretically justified basis of EWC. Moreover, we show that for SI the relation to the Fisher -- and in fact its performance -- is due to a previously unknown bias. On top of uncovering unknown similarities and unifying regularisation approaches, we also demonstrate that our insights enable practical performance improvements for large batch training.
Some machine learning applications require continual learning - where data comes in a sequence of datasets, each is used for training and then permanently discarded. From a Bayesian perspective, continual learning seems straightforward: Given the model posterior one would simply use this as the prior for the next task. However, exact posterior evaluation is intractable with many models, especially with Bayesian neural networks (BNNs). Instead, posterior approximations are often sought. Unfortunately, when posterior approximations are used, prior-focused approaches do not succeed in evaluations…
University of Amsterdam（阿姆斯特丹大学）；Vrije Universiteit（阿姆斯特丹自由大学）；Skoltech（斯科尔科沃科学技术学院）；AIRI（人工智能研究院）
Variational autoencoder (VAE) is a deep generative model for unsupervised learning, allowing to encode observations into the meaningful latent space. VAE is prone to catastrophic forgetting when tasks arrive sequentially, and only the data for the current one is available. We address this problem of continual learning for VAEs. It is known that the choice of the prior distribution over the latent space is crucial for VAE in the non-continual setting. We argue that it can also be helpful to avoid catastrophic forgetting. We learn the approximation of the aggregated posterior as a prior for each…
University of Technology Sydney（悉尼科技大学）；University of Cambridge（剑桥大学）；École Polytechnique Fédérale de Lausanne（洛桑联邦理工学院）；RIKEN Center for AI Project（理化学研究所人工智能项目中心）；University of Tübingen（图宾根大学）
Continually learning new skills is important for intelligent systems, yet standard deep learning methods suffer from catastrophic forgetting of the past. Recent works address this with weight regularisation. Functional regularisation, although computationally expensive, is expected to perform better, but rarely does so in practice. In this paper, we fix this issue by using a new functional-regularisation approach that utilises a few memorable past examples crucial to avoid forgetting. By using a Gaussian Process formulation of deep networks, our approach enables training in weight-space while …
Singapore Management University（新加坡管理大学）；Salesforce Research Asia（Salesforce亚洲研究院）
Continual learning aims to learn continuously from a stream of tasks and data in an online-learning fashion, being capable of exploiting what was learned previously to improve current and future tasks while still being able to perform well on the previous tasks. One common limitation of many existing continual learning methods is that they often train a model directly on all available training data without validation due to the nature of continual learning, thus suffering poor generalization at test time. In this work, we present a novel framework of continual learning named "Bilevel Continual…
Unifying Detection and Adaptation in Task-Free Continual Learning
School of Control Science and Engineering, Shandong University（山东大学控制科学与工程学院）；The Hong Kong University of Science and Technology（香港科技大学）；Shandong University（山东大学）；Qilu Hospital of Shandong University（山东大学齐鲁医院）
DeepMind（深度思维）；Gatsby Computational Neuroscience Unit, UCL（伦敦大学学院盖茨比计算神经科学中心）
We introduce a framework for Continual Learning (CL) based on Bayesian inference over the function space rather than the parameters of a deep neural network. This method, referred to as functional regularisation for Continual Learning, avoids forgetting a previous task by constructing and memorising an approximate posterior belief over the underlying task-specific function. To achieve this we rely on a Gaussian process obtained by treating the weights of the last layer of a neural network as random and Gaussian distributed. Then, the training algorithm sequentially encounters tasks and constru…
Convergence of Continual Learning in Homogeneous Deep Networks
Blavatnik School of Computer Science and AI, Tel Aviv University（塔夫茨大学Blavatnik计算机科学与人工智能学院）；Princeton University（普林斯顿大学）；Department of Electrical and Computing Engineering, Technion（技术学院电子与计算工程系）
Indian Institute of Technology（印度理工学院）；Deakin University（迪肯大学）
Humans learn continually throughout their lifespan by accumulating diverse knowledge and fine-tuning it for future tasks. When presented with a similar goal, neural networks suffer from catastrophic forgetting if data distributions across sequential tasks are not stationary over the course of learning. An effective approach to address such continual learning (CL) problems is to use hypernetworks which generate task dependent weights for a target network. However, the continual learning performance of existing hypernetwork based approaches are affected by the assumption of independence of the w…
An Inclusive and Lightweight Approach to Federated Continual Learning for Cultural Heritage
Information Technologies Institute, Centre for Research and Technology Hellas（希腊研究与技术中心信息技术研究所）；School of Electronic Engineering and Computer Science, Queen Mary University of London（伦敦大学玛丽女王学院电子工程与计算机科学学院）
Modern machine learning systems need to be able to cope with constantly arriving and changing data. Two main areas of research dealing with such scenarios are continual learning and data stream mining. Continual learning focuses on accumulating knowledge and avoiding forgetting, assuming information once learned should be stored. Data stream mining focuses on adaptation to concept drift and discarding outdated information, assuming that only the most recent data is relevant. While these two areas are mainly being developed in separation, they offer complementary views on the problem of learnin…
A Theory of Initialisation's Impact on Specialisation
School of Computer Science and Applied Mathematics, University of the Witwatersrand（威特沃特斯兰德大学计算机科学与应用数学学院）；Gatsby Computational Neuroscience Unit & Sainsbury Wellcome Centre, UCL（伦敦大学学院盖茨比计算神经科学单元与桑斯伯里惠康中心）；Machine Intelligence and Neural Discovery Institute, University of the Witwatersrand（威特沃特斯兰德大学机器智能与神经发现研究所）；CIFAR（加拿大高级研究所）
Enhancing Continual Learning for Software Vulnerability Prediction: Addressing Catastrophic Forgetting via Hybrid-Confidence-Aware Selective Replay for Temporal LLM Fine-Tuning
School of Computer Science, University of Nottingham, Nottingham, United Kingdom（诺丁汉大学计算机科学学院）；School of Informatics and Computing, Northern Arizona University, Flagstaff, United States of America（北亚利桑那大学信息学院）；School of Computer Science, University of Nottingham, Ningbo, China（诺丁汉大学宁波校区）
