通过对齐模型合并实现视觉-语言模型中的持续学习
Continual Learning in Vision-Language Models via Aligned Model Merging
AI总结：
本文提出基于对齐模型合并的持续学习方法，通过将新任务参数与先前参数对齐合并，在大型视觉-语言模型中平衡可塑性与稳定性，有效减少遗忘并提升泛化能力。
摘要：
持续学习通常通过顺序微调来解决，这一过程虽然能够实现适应，但在本质上更偏向可塑性，而牺牲了保留先前知识所需的稳定性。尽管现有方法试图缓解灾难性遗忘，但由于它们建立在顺序更新的基础上，对近期任务的偏置仍然存在。本文提出了一种基于模型合并的新视角，在保持可塑性的同时维持稳定性。我们不是仅仅顺序更新模型权重，而是提出将新训练的任务参数与先前学习的参数进行合并，从而促进更好的平衡。为了最大化合并过程的有效性，我们提出了一种简单机制，促进学习与先前权重对齐的权重，从而避免合并时的干扰。我们在大型视觉-语言模型（VLMs）上评估了该方法，并证明其在减少遗忘、提高对各种任务顺序和任务相似性的鲁棒性以及改善泛化能力方面的有效性。
英文摘要：
Continual learning is conventionally tackled through sequential fine-tuning, a process that, while enabling adaptation, inherently favors plasticity over the stability needed to retain prior knowledge. While existing approaches attempt to mitigate catastrophic forgetting, a bias towards recent tasks persists as they build upon this sequential nature. In this work we present a new perspective based on model merging to maintain stability while still retaining plasticity. Rather than just sequentially updating the model weights, we propose merging newly trained task parameters with previously learned ones, promoting a better balance. To maximize the effectiveness of the merging process, we propose a simple mechanism that promotes learning aligned weights with previous ones, thereby avoiding interference when merging. We evaluate this approach on large Vision-Language Models (VLMs), and demonstrate its effectiveness in reducing forgetting, increasing robustness to various task orders and similarities, and improving generalization.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
