To cope with real-world dynamics, an intelligent system needs to incrementally acquire, update, accumulate, and exploit knowledge throughout its lifetime. This ability, known as continual learning, provides a foundation for AI systems to develop themselves adaptively. In a general sense, continual learning is explicitly limited by catastrophic forgetting, where learning a new task usually results in a dramatic performance degradation of the old tasks. Beyond this, increasingly numerous advances have emerged in recent years that largely extend the understanding and application of continual learning. The growing and widespread interest in this direction demonstrates its realistic significance as well as complexity. In this work, we present a comprehensive survey of continual learning, seeking to bridge the basic settings, theoretical foundations, representative methods, and practical applications. Based on existing theoretical and empirical results, we summarize the general objectives of continual learning as ensuring a proper stability-plasticity trade-off and an adequate intra/inter-task generalizability in the context of resource efficiency. Then we provide a state-of-the-art and elaborated taxonomy, extensively analyzing how representative methods address continual learning, and how they are adapted to particular challenges in realistic applications. Through an in-depth discussion of promising directions, we believe that such a holistic perspective can greatly facilitate subsequent exploration in this field and beyond.
Taming Modality Entanglement in Continual Audio-Visual Segmentation
School of Artificial Intelligence, UCAS(人工智能学院,UCAS);MAIS, Institute of Automation(自动化研究所MAIS);School of Intelligent Science and Technology, University of Science and Technolog Beijing(智能科学与技术学院,北京理工大学)
Scaling Continual Learning to 300+ Tasks with Bi-Level Routing Mixture-of-Experts
School of Computing and Data Science, The University of Hong Kong(计算与数据科学学院,香港大学);Center for Embodied Artificial Intelligence and Computer Vision, Shenzhen Loop Area Institute(具身人工智能与计算机视觉中心,深圳环宇区研究所)
MCITlib: Multimodal Continual Instruction Tuning Library and Benchmark
School of Advanced Interdisciplinary Sciences, University of Chinese Academy of Sciences(中国科学院大学先进交叉学科学院);State Key Laboratory of Multimodal Artificial Intelligence Systems, Institute of Automation, Chinese Academy of Sciences(中国科学院自动化研究所多模态人工智能系统国家重点实验室);Centre for Artificial Intelligence and Robotics, Hong Kong Institute of Science and Innovation, Chinese Academy of Sciences(中国科学院香港创新科学研究院人工智能与机器人中心);School of Artificial Intelligence, University of Chinese Academy of Sciences(中国科学院大学人工智能学院);Fujian Key Laboratory of Pattern Recognition and Image Understanding, School of Computer and Information Engineering, Xiamen University of Technology(福建 pattern recognition and image understanding 工程学院,厦门大学科技学院)
Dynamic Sub-graph Distillation for Robust Semi-supervised Continual Learning
Tianjin University(天津大学);College of Intelligence and Computing, Tianjin University(天津大学智能与计算学部);Haihe Laboratory of Information Technology Application Innovation(海河信息技术应用创新实验室)
A Survey of Incremental Transfer Learning: Combining Peer-to-Peer Federated Learning and Domain Incremental Learning for Multicenter Collaboration
University Hospital Erlangen(埃尔朗根大学医院);Friedrich-Alexander-Universität Erlangen-Nürnberg(埃尔朗根-纽伦堡弗里德里希-亚历山大大学);Comprehensive Cancer Center Erlangen-EMN(埃尔朗根-EMN综合癌症中心);Bavarian Cancer Research Center(巴伐利亚癌症研究中心)
NeuCME: Toward Dynamic Multimodal Continual Learning via Neural Combinatorics of Multiple Experts
College of Computer Science, Sichuan University(四川大学计算机学院);School of Artificial Intelligence, Sichuan University(四川大学人工智能学院);Tianfu Jincheng Laboratory(天府锦城实验室);School of Economics and Management, China University of Petroleum (Beijing)(中国石油大学(北京)经济管理学院)
Overcoming Catastrophic Forgetting in Visual Continual Learning with Reinforcement Fine-Tuning
The University of Hong Kong(香港大学);The Hong Kong University of Science and Technology(香港科学与技术大学);Hong Kong Generative AI Research and Development Center(香港生成式人工智能研究与开发中心)
Continual Learning for Generative AI: From LLMs to MLLMs and Beyond
School of Advanced Interdisciplinary Sciences, University of Chinese Academy of Sciences(中国科学院大学先进交叉学科学院);MAIS, Institute of Automation, Chinese Academy of Sciences(中国科学院自动化研究所MAIS);Centre for Artificial Intelligence and Robotics, Hong Kong Institute of Science and Innovation, Chinese Academy of Sciences(中国科学院香港科学与创新研究院人工智能与机器人中心);School of Computer and Information Engineering, Xiamen University of Technology(厦门理工大学计算机与信息工程学院)
A Continual Learning-driven Model for Accurate and Generalizable Segmentation of Clinically Comprehensive and Fine-grained Whole-body Anatomies in CT
DAMO Academy, Alibaba Group(阿里巴巴达摩院);The first Affiliated Hospital, Zhejiang University(浙江大学附属第一医院);Hupan Laboratory(湖畔实验室);School of Medicine, Johns Hopkins University(约翰斯·霍普金斯大学医学院);Chang Gung Memorial Hospital(长庚纪念医院);Fudan University Shanghai Cancer Center(复旦大学附属肿瘤医院);Zhongshan Hospital, Fudan University(复旦大学附属中山医院);Sichuan Cancer Hospital(四川省肿瘤医院);Shanghai Institution of Pancreatic Disease(上海市胰腺疾病研究所);Xinhua Hospital Affiliated to Shanghai Jiaotong University School of Medicine(上海交通大学医学院附属新华医院);Linking Med Inc.(凌科医药);The University of Hong Kong(香港大学);University of Hong Kong–Shenzhen Hospital(香港大学深圳医院);Tan Tock Seng Hospital(陈笃生医院);Nanyang Technological University(南洋理工大学);Alibaba-NTU Global e-Sustainability CorpLab (ANGEL), Nanyang Technological University(南洋理工大学阿里巴巴-南洋理工全球电子可持续发展企业实验室);University of Wisconsin, Madison(威斯康星大学麦迪逊分校);National Institutes of Health Clinical Center(美国国立卫生研究院临床中心)
A Practitioner's Guide to Continual Multimodal Pretraining
Tübingen AI Center(蒂宾根人工智能中心);University of Tübingen(蒂宾根大学);Helmholtz Munich(亥姆霍兹慕尼黑中心);University of Cambridge(剑桥大学);LAION;Jülich Supercomputing Center (JSC)(于利希超级计算中心);Research Center Jülich (FZJ)(于利希研究中心);Helmholtz Association(亥姆霍兹联合会);Google DeepMind(谷歌DeepMind);Munich Center for ML(慕尼黑机器学习中心);Technical University of Munich(慕尼黑工业大学)
Forging a Dynamic Memory: Retrieval-Guided Continual Learning for Generalist Medical Foundation Models
College of Intelligent Robotics and Advanced Manufacturing(智能机器人与先进制造学院);Fudan University(复旦大学);Fysics Intelligence Technologies Co., Ltd. (Fysics AI)(Fysics智能技术有限公司(Fysics AI));School of Computer Science and Engineering(计算机科学与工程学院);Central South University(中南大学)
Continual Test-Time Adaptation in Computer Vision: Methods, Benchmarks, and Future Directions
The University of Texas at Dallas(德克萨斯大学达拉斯分校);LIVIA ETS Montreal, ILLS International Laboratory on Learning Systems (ILLS)(蒙特利尔LIVIA ETS,学习系统国际实验室(ILLS));Tulane University(路易斯安那州立大学);Nanyang Technological University(南洋理工大学);Hanyang University(翰阳大学)
CMFDFormer: Transformer-based Copy-Move Forgery Detection with Continual Learning
Beijing Electronic Science and Technology Institute(北京电子科技学院);Jimei University(集美大学);Xidian University(西安电子科技大学);University of Science and Technology of China(中国科学技术大学)
ViDA: Homeostatic Visual Domain Adapter for Continual Test Time Adaptation
National Key Laboratory for Multimedia Information Processing(多媒体信息处理国家重点实验室);The Chinese University of Hong Kong(香港中文大学);Hong Kong University of Science and Technology(香港科技大学);School of Computer Science Peking University(北京大学计算机学院);AIRobotics
Information-Theoretic Generalization Bounds of Replay-based Continual Learning
School of Computer Science and Technology, Xi’an Jiaotong University(西安交通大学计算机科学与技术学院);Department of Oncology, University of Cambridge(剑桥大学肿瘤学系);China Telecom, China(中国电信);Department of Computer Science and Technology, Tsinghua University(清华大学计算机科学与技术系)
CL-VISTA: Benchmarking Continual Learning in Video Large Language Models
School of Advanced Interdisciplinary Sciences, University of Chinese Academy of Sciences(中国科学院大学前沿交叉科学学院);MAIS, Institute of Automation, Chinese Academy of Sciences(中国科学院自动化研究所多模态人工智能系统实验室);School of Artificial Intelligence, University of Chinese Academy of Sciences(中国科学院大学人工智能学院);FKLPRIU, School of Computer and Information Engineering, Xiamen University of Technology(厦门理工学院计算机与信息工程学院福建省模式识别与图像理解重点实验室)
Continual Hand-Eye Calibration for Open-world Robotic Manipulation
School of Automation Science and Engineering, South China University of Technology(华南理工大学自动化科学与工程学院);State Key Laboratory of Robotics, the Institutes for Robotics and Intelligent Manufacturing, Chinese Academy of Sciences(中国科学院机器人研究所);University of Chinese Academy of Sciences(中国科学院大学)