arXiv 2605.149382026-05-14cs.LGcs.CV

Octopus：无历史梯度正交化用于多模态大语言模型中的持续学习

Octopus: History-Free Gradient Orthogonalization for Continual Learning in Multimodal Large Language Models

MoE Key Lab of Artificial Intelligence, AI Institute, Shanghai Jiao Tong University（人工智能大模型关键实验室，人工智能研究院，上海交通大学）；vivo Mobile Communication Co., Ltd.（vivo移动通信有限公司）

Yuehao Liu, Shanyan Guan, Weijia Zhang, Xuanming Shang, Yanhao Ge, Wei Li, Chao Ma

阅读 PDF

arXiv 页面 ↗HTML 原文 ↗

AI总结：

Octopus提出一种基于无历史梯度正交化的方法，通过两阶段微调策略平衡持续学习中的可塑性与稳定性，实验表明其在UCIT数据集上性能优于现有最佳方法。

摘要：

英文摘要：

Continual learning in multimodal large language models (MLLMs) aims to sequentially acquire knowledge while mitigating catastrophic forgetting, yet existing methods face inherent limitations: architecture-based approaches incur additional computational overhead and often generalize poorly to new tasks, rehearsal-based methods rely on storing historical data, raising privacy and storage concerns, and conventional regularization-based strategies alone are insufficient to fully prevent parameter interference. We propose Octopus, a two-stage continual learning framework based on History-Free Gradient Orthogonalization (HiFGO), which enforces gradient-level orthogonality without historical task data. Our proposed two-stage finetuning strategy decouples task adaptation from regularization, achieving a principled balance between plasticity and stability. Experiments on UCIT show that Octopus establishes state-of-the-art performance, surpassing prior SOTA by 2.14% and 6.82% in terms of Avg and Last.

机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。