arXiv 2508.073072025-12-31cs.CVcs.AI

MCITlib: 多模态持续指令微调库与基准

MCITlib: Multimodal Continual Instruction Tuning Library and Benchmark

School of Advanced Interdisciplinary Sciences, University of Chinese Academy of Sciences（中国科学院大学先进交叉学科学院）；State Key Laboratory of Multimodal Artificial Intelligence Systems, Institute of Automation, Chinese Academy of Sciences（中国科学院自动化研究所多模态人工智能系统国家重点实验室）；Centre for Artificial Intelligence and Robotics, Hong Kong Institute of Science and Innovation, Chinese Academy of Sciences（中国科学院香港创新科学研究院人工智能与机器人中心）；School of Artificial Intelligence, University of Chinese Academy of Sciences（中国科学院大学人工智能学院）；Fujian Key Laboratory of Pattern Recognition and Image Understanding, School of Computer and Information Engineering, Xiamen University of Technology（福建 pattern recognition and image understanding 工程学院，厦门大学科技学院）

Haiyang Guo, Fei Zhu, Hongbo Zhao, Fanhu Zeng, Wenzhuo Liu, Shijie Ma, Da-Han Wang, Xu-Yao Zhang

备注：: Preprint

阅读 PDF

arXiv 页面 ↗HTML 原文 ↗

AI总结：

MCITlib提供多模态持续学习的库和基准，支持8种算法并评估3个基准，旨在解决灾难性遗忘和跨模态协调问题。

摘要：

持续学习使AI系统能够在保留之前学习信息的同时获取新知识。尽管传统单模态方法取得进展，但多模态大语言模型（MLLMs）的兴起给多模态持续学习（MCL）带来了新挑战，其中模型需同时解决灾难性遗忘和跨模态协调问题。为推动该领域研究，我们提出了MCITlib，一个全面的多模态持续指令微调库。MCITlib目前实现了8种代表性算法，并在2种主干模型下对3个基准进行评估。该库将持续更新以支持MCL未来的发展。代码库发布于https://github.com/Ghy0501/MCITlib。

英文摘要：

Continual learning enables AI systems to acquire new knowledge while retaining previously learned information. While traditional unimodal methods have made progress, the rise of Multimodal Large Language Models (MLLMs) brings new challenges in Multimodal Continual Learning (MCL), where models are expected to address both catastrophic forgetting and cross-modal coordination. To advance research in this area, we present MCITlib, a comprehensive library for Multimodal Continual Instruction Tuning. MCITlib currently implements 8 representative algorithms and conducts evaluations on 3 benchmarks under 2 backbone models. The library will be continuously updated to support future developments in MCL. The codebase is released at https://github.com/Ghy0501/MCITlib.

机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。