BECAME：基于自适应模型合并的贝叶斯持续学习
BECAME: BayEsian Continual Learning with Adaptive Model MErging
AI总结：
本文提出BECAME框架，基于贝叶斯持续学习原理推导出任务自适应的最优模型合并系数闭式解，结合梯度投影与自适应合并，有效平衡稳定性与可塑性，显著优于现有持续学习和模型合并方法。
摘要：
持续学习（CL）致力于跨任务增量学习，同时缓解灾难性遗忘。CL中的一个关键挑战是平衡稳定性（保留先验知识）和可塑性（学习新任务）。具有代表性的梯度投影方法虽能确保稳定性，但往往限制了可塑性。模型合并技术提供了有前景的解决方案，但现有方法通常依赖经验假设和精心选择的超参数。本文探索了模型合并增强稳定性与可塑性权衡的潜力，并提供了强调其优势的理论见解。具体而言，我们利用贝叶斯持续学习原理重新表述合并机制，并推导出适应任务多样特征的最优合并系数闭式解。为验证该方法，我们提出了名为BECAME的两阶段框架，协同了梯度投影与自适应合并的专长。大量实验表明，该方法优于最先进的CL方法和现有合并策略。
英文摘要：
Continual Learning (CL) strives to learn incrementally across tasks while mitigating catastrophic forgetting. A key challenge in CL is balancing stability (retaining prior knowledge) and plasticity (learning new tasks). While representative gradient projection methods ensure stability, they often limit plasticity. Model merging techniques offer promising solutions, but prior methods typically rely on empirical assumptions and carefully selected hyperparameters. In this paper, we explore the potential of model merging to enhance the stability-plasticity trade-off, providing theoretical insights that underscore its benefits. Specifically, we reformulate the merging mechanism using Bayesian continual learning principles and derive a closed-form solution for the optimal merging coefficient that adapts to the diverse characteristics of tasks. To validate our approach, we introduce a two-stage framework named BECAME, which synergizes the expertise of gradient projection and adaptive merging. Extensive experiments show that our approach outperforms state-of-the-art CL methods and existing merging strategies.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
