多模态持续学习的最新进展：综合性综述
Recent Advances of Multimodal Continual Learning: A Comprehensive Survey
AI总结：
本文综述了多模态持续学习的最新进展，分析了其核心挑战与方法分类，提出四种主要方法类别，并讨论了未来研究方向和开放资源。
摘要：
持续学习（CL）旨在使机器学习模型能够从新数据中持续学习，同时在不遗忘之前知识的基础上进行学习。随着模型从小型到大型预训练架构的发展，以及从支持单模态到多模态数据的演变，多模态持续学习（MMCL）方法最近出现。MMCL的主要复杂性在于它扩展了单模态CL方法的简单堆叠。此类直接方法常导致多模态灾难性遗忘，导致性能不佳。此外，MMCL引入了单模态CL方法无法充分解决的新挑战，包括模态不平衡、复杂的模态交互、高计算成本以及多模态主干的预训练零样本能力退化。在本文中，我们提出了首个关于MMCL的综合性综述。我们提供了必要的背景知识和MMCL设置，以及MMCL方法的结构化分类。我们将MMCL方法分为四类，即基于正则化的、基于架构的、基于重放的和基于提示的方法，解释了它们的方法论并突出了其关键创新。此外，为了促进该领域进一步研究，我们总结了开放的MMCL数据集和基准，提供了深入的讨论，并讨论了几个有前途的未来方向。我们还创建了一个GitHub仓库来索引相关的MMCL论文和开放资源，网址为https://github.com/LucyDYu/Awesome-Multimodal-Continual-Learning。
英文摘要：
Continual learning (CL) aims to empower machine learning models to learn continually from new data, while building upon previously acquired knowledge without forgetting. As models have evolved from small to large pre-trained architectures, and from supporting unimodal to multimodal data, multimodal continual learning (MMCL) methods have recently emerged. The primary complexity of MMCL is that it extends beyond a simple stacking of unimodal CL methods. Such straightforward approaches often suffer from multimodal catastrophic forgetting, yielding unsatisfactory performance. In addition, MMCL introduces new challenges that unimodal CL methods fail to adequately address, including modality imbalance, complex modality interaction, high computational costs, and degradation of pre-trained zero-shot capability of multimodal backbones. In this work, we present the first comprehensive survey on MMCL. We provide essential background knowledge and MMCL settings, as well as a structured taxonomy of MMCL methods. We categorize MMCL methods into four categories, i.e., regularization-based, architecture-based, replay-based, and prompt-based methods, explaining their methodologies and highlighting their key innovations. Additionally, to prompt further research in this field, we summarize open MMCL datasets and benchmarks, provide an in-depth discussion, and discuss several promising future directions. We have also created a GitHub repository for indexing relevant MMCL papers and open resources available at https://github.com/LucyDYu/Awesome-Multimodal-Continual-Learning.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
