视觉语言模型（VLMs）和近期多模态大语言模型（MLLMs）通过前所未有的跨模态对齐和零样本泛化革新了人工智能。然而，使它们能够从非平稳数据中持续学习仍是一个重大挑战，因为它们的跨模态对齐和泛化能力特别容易受到灾难性遗忘的影响。不同于传统单模态持续学习（CL），VLMs面临独特的挑战，如跨模态特征漂移、由于共享架构导致的参数干扰以及零样本能力侵蚀。此外，生成式MLLMs表现出一种独特的“对齐税”，其中灾难性遗忘不仅表现为事实性遗忘，还表现为深度链式思维（CoT）推理的系统性崩溃。本文首次全面、诊断性地回顾了预测VLMs和生成式MLLMs的持续学习。我们系统地分解了上述失败模式，并提出了一个以挑战为导向的分类，包括四个核心范式：（1）多模态重播策略解决显式和隐式记忆漂移；（2）跨模态正则化强制拓扑和几何对齐；（3）参数高效适应利用动态路由和子空间投影；以及新兴的（4）模型融合与解耦范式。我们批判性地分析了评估协议的演变，强调了向双轨基准（领域 vs. 能力 CL）和微诊断 CoT 评估的转变。最后，我们绘制了未来研究的路线图，强调组合式零样本学习、具身AI与传感器融合以及自主智能体生态系统。所有资源均可在：https://github.com/YuyangSunshine/Awesome-Continual-learning-of-Vision-Language-Models 上找到。
英文摘要：
Vision-language models (VLMs), spanning predictive architectures to generative Multimodal Large Language Models (MLLMs), have revolutionized artificial intelligence through powerful cross-modal alignment and zero-shot generalization. However, enabling them to learn continually from non-stationary data remains a major challenge, as their cross-modal alignment and generalization capabilities are particularly vulnerable to catastrophic forgetting. Unlike traditional unimodal continual learning (CL), VLMs face unique challenges such as cross-modal feature drift, parameter interference due to shared architectures, and zero-shot capability erosion. Furthermore, generative MLLMs exhibit a unique "alignment tax," where catastrophic forgetting manifests not merely as factual amnesia, but as a systemic collapse of deep Chain-of-Thought (CoT) reasoning. This survey presents the first comprehensive diagnostic review bridging continual learning across predictive VLMs and generative MLLMs. We systematically deconstruct the aforementioned failure modes and propose a challenge-driven taxonomy comprising four core paradigms: (1) Multi-Modal Replay Strategies addressing explicit and implicit memory drift; (2) Cross-Modal Regularization enforcing topological and geometric alignment; (3) Parameter-Efficient Adaptation utilizing dynamic routing and subspace projections; and the emerging (4) Model Fusion and Decoupling paradigms. We critically analyze the evolution of evaluation protocols, highlighting the essential shift toward dual-track benchmarks (Domain vs. Ability CL). Finally, we chart a roadmap for future research, emphasizing compositional zero-shot learning, embodied AI with sensor fusion, and autonomous agentic ecosystems. All resources are available at: https://github.com/YuyangSunshine/Awesome-Continual-learning-of-Vision-Language-Models
Tianjin University（天津大学）；Shanghai Artificial Intelligence Laboratory（上海人工智能实验室）；The University of Sheffield（谢菲尔德大学）
Vision Language Models (VLMs), pre-trained on large-scale image-text datasets, enable zero-shot predictions for unseen data but may underperform on specific unseen tasks. Continual learning (CL) can help VLMs effectively adapt to new data distributions without joint training, but faces challenges of catastrophic forgetting and generalization forgetting. Although significant progress has been achieved by distillation-based methods, they exhibit two severe limitations. One is the popularly adopted single-teacher paradigm fails to impart comprehensive knowledge, The other is the existing methods …
Bisecle: Binding and Separation in Continual Learning for Video Language Understanding
School of Computer Science University of New South Wales（计算机科学学院新南威尔士大学）；School of Computer Science and Engineering University of New South Wales（计算机科学与工程学院新南威尔士大学）；DEVCOM Army Research Laboratory（陆军研究实验室）
Computer Vision Center, UAB（巴塞罗那自治大学计算机视觉中心）；Huawei（华为）
Artificial neural networks thrive in solving the classification problem for a particular rigid task, acquiring knowledge through generalized learning behaviour from a distinct training phase. The resulting network resembles a static entity of knowledge, with endeavours to extend this knowledge without targeting the original task resulting in a catastrophic forgetting. Continual learning shifts this paradigm towards networks that can continually accumulate knowledge over different tasks without the need to retrain from scratch. We focus on task incremental classification, where tasks arrive seq…
Advancing Cross-domain Discriminability in Continual Learning of Vision-Language Models
Institute of Science Tokyo（东京科学研究院）；University of California Berkeley（加州大学伯克利分校）；Nanyang Technological University（南洋理工大学）；South China University of Technology（华南理工大学）；Greater Bay Area Institute for Innovation, Hunan University（湖南大学粤港澳大湾区创新研究院）
School of Computer Science and Engineering, Macau University of Science and Technology（澳门科技大学计算机科学与工程学院）；Department of Computer Science, Hong Kong Baptist University（香港浸会大学计算机科学系）
Beijing University of Posts and Telecommunications（北京邮电大学）；Shanghai Artificial Intelligence Laboratory（上海人工智能实验室）；West China Hospital, Sichuan University（四川大学华西医院）；Shanghai Jiao Tong University（上海交通大学）；Sichuan University Pittsburgh Institute（四川大学匹兹堡学院）
Inevitable domain and task discrepancies in real-world scenarios can impair the generalization performance of the pre-trained deep models for medical data. Therefore, we audaciously propose that we should build a general-purpose medical AI system that can be seamlessly adapted to downstream domains/tasks. Since the domain/task adaption procedures usually involve additional labeling work for the target data, designing a data-efficient adaption algorithm is desired to save the cost of transferring the learned knowledge. Our recent work found that vision-language models (VLMs) are efficient learn…
Adapting Vision-Language Models Without Labels: A Comprehensive Survey
ETH Zürich（苏黎世联邦理工学院）；University of Science and Technology of China（中国科学技术大学）；Institute of Automation, Chinese Academy of Sciences（中国科学院自动化研究所）；EPFL（苏黎世联邦理工学院）
Large Language Models for Crash Detection in Video: A Survey of Methods, Datasets, and Challenges
Department of Computer Science, Iowa State University（计算机科学系，爱荷华州立大学）；Department of Civil, Construction and Environmental Engineering, Iowa State University（土木、建设与环境工程系，爱荷华州立大学）
Approximate Domain Unlearning for Vision-Language Models
Tokyo University of Science（东京科学大学）；National University of Singapore（新加坡国立大学）；National Institute of Advanced Industrial Science and Technology (AIST)（国家先进工业科学与技术研究院）；University of Oxford（牛津大学）
DOTA: Distributional Test-Time Adaptation of Vision-Language Models
State Key Laboratory of Networking and Switching Technology（网络与交换技术国家重点实验室）；Beijing University of Posts and Telecommunications（北京邮电大学）；College of Intelligence and Computing（智能与计算学院）；Tianjin University（天津大学）；School of Computer Science and Technology（计算机科学与技术学院）；Harbin Institute of Technology Shenzhen（哈尔滨工业大学深圳学院）；Institute for Infocomm Research, A*STAR（信息通信研究机构，A*STAR）；Show Lab, National University of Singapore（新加坡国立大学Show实验室）；National University of Singapore（新加坡国立大学）
CalFuse: Multi-Modal Continual Learning via Feature Calibration and Parameter Fusion
College of Intelligent Robotics and Advanced Manufacturing, Fudan University（智能机器人与先进制造学院，复旦大学）；School of Future Science and Engineering, Soochow University（未来科学与工程学院，苏州大学）；DataLab: Data Science and Informatics, University of California, Davis（数据实验室：数据科学与信息学，加州大学戴维斯分校）；University of Rochester（罗切斯特大学）；Ningbo University（宁波大学）；Hong Kong University of Science and Technology (Guangzhou)（香港科技大学（广州））；Beijing University of Posts and Telecommunications（北京邮电大学）；Academy for Computer Science and Informatics, Cardiff University（计算机科学与信息学学院，卡迪夫大学）
University of Science and Technology of China（中国科学技术大学）；Institute of Intelligent Machines, Chinese Academy of Sciences（中国科学院合肥智能机械研究所）；Mississippi State University（密西西比州立大学）；Sichuan University（四川大学）；Southwest Jiaotong University（西南交通大学）；Central South University（中南大学）
Humans' continual learning (CL) ability is closely related to Stability Versus Plasticity Dilemma that describes how humans achieve ongoing learning capacity and preservation for learned information. The notion of CL has always been present in artificial intelligence (AI) since its births. This paper proposes a comprehensive review of CL. Different from previous reviews that mainly focus on the catastrophic forgetting phenomenon in CL, this paper surveys CL from a more macroscopic perspective based on the Stability Versus Plasticity mechanism. Analogous to biological counterpart, "smart" AI ag…
