带扰动训练、合并后推理：一种用于持续学习的两阶段框架
Train with Perturbation, Infer after Merging: A Two-Stage Framework for Continual Learning
AI总结：
提出Perturb-and-Merge框架，通过训练后模型合并与基于Hessian的正则化及随机扰动，结合LoRA，在持续学习中缓解灾难性遗忘，达到最先进性能。
摘要：
持续学习（CL）旨在使模型能够从一系列任务中持续获取新知识，同时避免遗忘已学到的信息。然而，现有的持续学习方法仅依赖最近任务的参数进行推理，这使得它们容易遭受灾难性遗忘。受模型合并技术近期成功的启发，我们提出了Perturb-and-Merge（P&M），一种新颖的持续学习框架，将模型合并集成到持续学习范式中以缓解遗忘。具体来说，在每个任务训练后，P&M通过形成先前模型与新训练的任务特定模型的凸组合来构建新模型。通过理论分析，我们最小化所有任务的总损失增加，并在温和假设下推导出合并系数的闭式解。为了进一步提升合并模型的性能，我们观察到合并过程中引入的退化可以通过由任务向量和损失函数的Hessian矩阵组成的正则化项来缓解。有趣的是，我们证明该项可以使用二阶对称有限差分高效近似，并据此设计了一种沿任务向量方向的随机扰动策略，该策略不产生额外的前向或反向传播，同时提供正则化项的有效近似。最后，我们将P&M与参数高效微调方法LoRA相结合，以减少内存开销。我们提出的方法在多个持续学习基准数据集上取得了最先进的性能。代码可在https://github.com/qhmiao/P-M-for-Continual-Learning获取。
英文摘要：
Continual Learning (CL) aims to enable models to continuously acquire new knowledge from a sequence of tasks with avoiding the forgetting of learned information. However, existing CL methods only rely on the parameters of the most recent task for inference, which makes them susceptible to catastrophic forgetting. Inspired by the recent success of model merging techniques, we propose \textbf{Perturb-and-Merge (P\&M)}, a novel continual learning framework that integrates model merging into the CL paradigm to mitigate forgetting. Specifically, after training on each task, P\&M constructs a new model by forming a convex combination of the previous model and the newly trained task-specific model. Through theoretical analysis, We minimize the total loss increase across all tasks and derive a closed-form solution for the merging coefficient under mild assumptions. To further improve the performance of the merged model, we observe that the degradation introduced during merging can be alleviated by a regularization term composed of the task vector and the Hessian matrix of the loss function. Interestingly, we show that this term can be efficiently approximated using second-order symmetric finite differences, and a stochastic perturbation strategy along the task vector direction is accordingly devised which incurs no additional forward or backward passes while providing an effective approximation of the regularization term. Finally, we combine P\&M with LoRA, a parameter-efficient fine-tuning method, to reduce memory overhead. Our proposed approach achieves state-of-the-art performance on several continual learning benchmark datasets. The code is available at https://github.com/qhmiao/P-M-for-Continual-Learning.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
