在持续学习之前进行适应
Adapt before Continual Learning
AI总结：
本文提出ACL框架，在持续学习前通过优化预训练模型主干以平衡稳定性与可塑性，提升持续学习性能。
摘要：
持续学习（CL）旨在使神经网络能够逐步获取新知识（可塑性）的同时保留已有知识（稳定性）。尽管预训练模型（PTMs）为CL提供了强有力的基石，但现有方法在平衡这两个竞争性目标时面临根本性挑战。当前方法通常通过冻结PTM主干来实现稳定性，这严重限制了模型的可塑性，尤其是在输入数据分布与预训练数据差异较大的情况下。另一种方法是依次微调整个PTM以适应新知识，但往往导致灾难性遗忘，突显了基于PTM的CL中稳定性与可塑性的关键权衡。为了解决这一限制，我们提出了在核心CL过程之前进行PTM适应（ACL）的新框架，该框架引入了一个即插即用的适应阶段，用于在学习每个新任务之前。在这一阶段，ACL通过将嵌入对齐其原始类原型并将其与无关类分离来优化PTM主干。这种机制理论和实验证明了稳定性与可塑性之间的良好平衡，显著提高了CL在基准和集成方法上的性能。代码可在https://github.com/byyx666/ACL_code获取。
英文摘要：
Continual Learning (CL) seeks to enable neural networks to incrementally acquire new knowledge (plasticity) while retaining existing knowledge (stability). Although pre-trained models (PTMs) have provided a strong foundation for CL, existing approaches face a fundamental challenge in balancing these two competing objectives. Current methods typically address stability by freezing the PTM backbone, which severely limits the model's plasticity, particularly when incoming data distribution diverges largely from the pre-training data. Alternatively, sequentially fine-tuning the entire PTM can adapt to new knowledge but often leads to catastrophic forgetting, highlighting the critical stability-plasticity trade-off in PTM-based CL. To address this limitation, we propose Adapting PTMs before the core CL} process (ACL), a novel framework that introduces a plug-and-play adaptation phase prior to learning each new task. During this phase, ACL refines the PTM backbone by aligning embeddings with their original class prototypes while distancing them from irrelevant classes. This mechanism theoretically and empirically demonstrates desirable balance between stability and plasticity, significantly improving CL performance across benchmarks and integrated methods. Code is available at https://github.com/byyx666/ACL_code.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
