MLLM-CL：面向多模态大语言模型的持续学习
MLLM-CL: Continual Learning for Multimodal Large Language Models
AI总结：
针对多模态大语言模型在动态场景下持续学习新知识技能的难题，本文提出包含领域与能力两类持续学习的MLLM-CL基准，以及基于参数隔离和MLLM路由的抗遗忘方法，实验证明其性能显著优于现有方法。
摘要：
近期的多模态大语言模型（MLLMs）在视觉-语言理解任务中表现出色，但在适应需要持续整合新知识与新技能的动态真实场景时面临挑战。尽管持续学习（CL）提供了潜在的解决方案，但现有基准与方法存在关键局限性。本文提出MLLM-CL，这是一个涵盖领域持续学习与能力持续学习的新型基准：前者聚焦于在不断演进的主流领域间开展独立同分布（IID）评估，后者则针对具备新模型能力的非独立同分布场景进行评估。在方法层面，我们提出通过参数隔离与基于MLLM的路由机制来防止灾难性干扰。大量实验表明，我们的方法能够以极低的遗忘程度整合领域特定知识与功能能力，性能显著优于现有方法。我们的基准与代码可在https://github.com/bjzhb666/MLLM-CL获取。
英文摘要：
Recent Multimodal Large Language Models (MLLMs) excel in vision-language understanding but face challenges in adapting to dynamic real-world scenarios that require continuous integration of new knowledge and skills. While continual learning (CL) offers a potential solution, existing benchmarks and methods suffer from critical limitations. In this paper, we introduce MLLM-CL, a novel benchmark encompassing domain and ability continual learning, where the former focuses on independently and identically distributed (IID) evaluation across evolving mainstream domains, whereas the latter evaluates on non-IID scenarios with new model abilities. Methodologically, we propose preventing catastrophic interference through parameter isolation and an MLLM-based routing mechanism. Extensive experiments demonstrate that our approach can integrate domain-specific knowledge and functional abilities with minimal forgetting, significantly outperforming existing methods. Our benchmark and code are available at https://github.com/bjzhb666/MLLM-CL.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
