超越保留：在连续学习中协调结构安全与可塑性
Beyond Retention: Orchestrating Structural Safety and Plasticity in Continual Learning for LLMs
AI总结：
本文提出OSW方法，通过正交子空间唤醒协调连续学习中的结构安全与可塑性，有效保留脆弱任务能力并维持新任务学习。
摘要：
在大型语言模型（LLMs）中，连续学习面临平衡稳定性（保留旧知识）和可塑性（学习新任务）的关键挑战。尽管经验回放（ER）是缓解灾难性遗忘的标准对策，但其在多样化能力上的影响仍待探索。在本工作中，我们揭示了ER行为中的关键二元对立：尽管它在稳健、无结构的任务上诱导正向反向转移（例如通过重复复习提升先前NLP分类任务的表现），但它在脆弱、结构化的领域如代码生成中导致严重的负向转移（例如编码准确性显著下降）。这表明ER以结构完整性换取了广泛的巩固。为解决这一困境，我们提出了\textbf{正交子空间唤醒（OSW）}。OSW通过简短的
英文摘要：
Continual learning in Large Language Models (LLMs) faces the critical challenge of balancing stability (retaining old knowledge) and plasticity (learning new tasks). While Experience Replay (ER) is a standard countermeasure against catastrophic forgetting, its impact across diverse capabilities remains underexplored. In this work, we uncover a critical dichotomy in ER's behavior: while it induces positive backward transfer on robust, unstructured tasks (e.g., boosting performance on previous NLP classification tasks through repeated rehearsal), it causes severe negative transfer on fragile, structured domains like code generation (e.g., a significant relative drop in coding accuracy). This reveals that ER trades structural integrity for broad consolidation. To address this dilemma, we propose \textbf{Orthogonal Subspace Wake-up (OSW)}. OSW identifies essential parameter subspaces of previous tasks via a brief "wake-up" phase and enforces orthogonal updates for new tasks, providing a mathematically grounded "safety guarantee" for established knowledge structures. Empirical results across a diverse four-task sequence demonstrate that OSW uniquely succeeds in preserving fragile coding abilities where Replay fails, while simultaneously maintaining high plasticity for novel tasks. Our findings emphasize the necessity of evaluating structural safety alongside average retention in LLM continual learning.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
