In Continual Learning (CL), a neural network is trained on a stream of data whose distribution changes over time. In this context, the main problem is how to learn new information without forgetting old knowledge (i.e., Catastrophic Forgetting). Most existing CL approaches focus on finding solutions to preserve acquired knowledge, so working on the past of the model. However, we argue that as the model has to continually learn new tasks, it is also important to put focus on the present knowledge that could improve following tasks learning. In this paper we propose a new, simple, CL algorithm that focuses on solving the current task in a way that might facilitate the learning of the next ones. More specifically, our approach combines the main data stream with a secondary, diverse and uncorrelated stream, from which the network can draw auxiliary knowledge. This helps the model from different perspectives, since auxiliary data may contain useful features for the current and the next tasks and incoming task classes can be mapped onto auxiliary classes. Furthermore, the addition of data to the current task is implicitly making the classifier more robust as we are forcing the extraction of more discriminative features. Our method can outperform existing state-of-the-art models on the most common CL Image Classification benchmarks.
Improving Continual Learning Performance and Efficiency with Auxiliary Classifiers
Warsaw University of Technology(华沙技术大学);Computer Vision Center, Barcelona(巴塞罗那计算机视觉中心);IDEAS Research Institute(IDEAS研究机构);Institute of Artificial Intelligence and Robotics(人工智能与机器人研究所);Xi’an Jiaotong University(西安交通大学);Nankai University(南开大学)
Online Fast Adaptation and Knowledge Accumulation: a New Approach to Continual Learning
ElementAI(ElementAI公司);Mila - Quebec AI Institute(米拉-魁北克人工智能研究所);Université de Montréal(蒙特利尔大学);Facebook AI Research(脸书人工智能研究院);McGill University(麦吉尔大学);HEC Montréal(蒙特利尔高等商学院)
Federated Continual Learning via Knowledge Fusion: A Survey
Nanyang Technological University(南洋理工大学);JD iCity, JD Technology(京东科技京东智城);JD Intelligent Cities Research & Institute of Artificial Intelligence, Southwest Jiaotong University(西南交通大学京东智慧城市研究院与人工智能研究所);School of Computing and Artificial Intelligence, Southwest Jiaotong University(西南交通大学计算机与人工智能学院)
The Future of Continual Learning in the Era of Foundation Models: Three Key Directions
Department of Computer Science, Università di Pisa(帕尔马大学计算机科学系);Institute of Information Science and Technologies, National Research Council(信息科学与技术研究所,国家研究理事会)
Exploring Open-world Continual Learning with Knowns-Unknowns Knowledge Transfer
Southwestern University of Finance and Economics(西南财经大学);Leiden University(莱顿大学);School of Computing and Artificial Intelligence, Southwest Jiaotong University(西南交通大学计算机与人工智能学院)
Knowledge Capture and Replay for Continual Learning
Artificial Intelligence, Analytics And Informatics (AI), A*STAR(A*STAR人工智能、分析与信息学);RMIT University(皇家墨尔本理工大学);Institute for Infocomm Research (IR), A*STAR(A*STAR资讯通信研究院)
Prototype-Guided Non-Exemplar Continual Learning for Cross-subject EEG Decoding
Dept. of Artificial Intelligence Korea University Seoul, Republic of Korea(人工智能系 韩国大学首尔共和国);Cognitive Engineering Korea University Seoul, Republic of Korea(认知工程 韩国大学首尔共和国)
CMKL: Modality-Aware Continual Learning for Evolving Biomedical Knowledge Graphs
Technology, Innovation, Entrepreneurship Department(技术、创新与创业部门);King Abdullah University of Science and Technology(卡塔尔科技大学);School of Computing and Information Systems(计算与信息系统学院);The University of Melbourne(墨尔本大学);College of Computer Science and Technology(计算机科学与技术学院);Jilin University(吉林大学);School of Computing Technologies(计算技术学院);RMIT University(皇家墨尔本理工大学);College of Computing and Data Science(计算与数据科学学院);Nanyang Technological University(南洋理工大学)
College of Computer Science and Technology, Zhejiang University(浙江大学计算机科学与技术学院);ZJU-Ant Group Joint Research Center for Knowledge Graphs, Zhejiang University(浙江大学-蚂蚁集团知识图谱联合研究中心);School of Software Technology, Zhejiang University(浙江大学软件学院);Monash University(莫纳什大学);Hangzhou Dianzi University(杭州电子科技大学);Zhejiang Lab(之江实验室)
Continual-NExT: A Unified Comprehension And Generation Continual Learning Framework
East China Normal University(华东师范大学);Shanghai Innovation Institute(上海创新研究院);Shanghai Artificial Intelligence Laboratory(上海人工智能实验室);Xiamen University(厦门大学)
Rehearsal-Free Modular and Compositional Continual Learning for Language Models
Bosch Center for Artificial Intelligence(博世人工智能中心);LMU Munich(慕尼黑大学);Hochschule der Medien(斯图加特媒体大学);Karlsruhe University of Applied Sciences(卡尔斯鲁厄应用技术大学)
Semi-parametric Memory Consolidation: Towards Brain-like Deep Continual Learning
Centre for Artificial Intelligence and Robotics(人工智能与机器人中心);Hong Kong Institute of Science and Innovation(香港科学与创新研究院);Chinese Academy of Sciences(中国科学院);Department of Computer Science(计算机科学系);City University of Hong Kong(香港城市大学);The Department of Neurosurgery(神经外科部门);Peking University First Hospital(北京大学第一医院);State Key Laboratory of Multimodal Artificial Intelligence Systems(多模态人工智能系统国家重点实验室);Institute of Automation(自动化研究所);Hong Kong SAR, China(香港特别行政区,中国)