灾难性遗忘是低秩的：持续适应的函数空间理论
Catastrophic Forgetting is Low-Rank: A Function-Space Theory for Continual Adaptation
AI总结：
本文在神经正切核（NTK）框架下提出函数空间理论，推导出新任务训练导致旧任务预测漂移的闭式表达式，揭示遗忘集中在少量旧任务NTK本征模式上，并给出低秩特性与Kronecker缩放规则。
摘要：
持续适应中的灾难性遗忘通常通过参数漂移、重放或蒸馏来研究，但这些观点未能识别哪些输出空间方向是脆弱的。我们在NTK机制下给出一个函数空间解释：新任务训练通过跨任务核诱导旧任务预测漂移，从而在新任务梯度步骤之前得到遗忘向量的闭式预测器。在冻结主干线性头PEFT-CL中，模型在可训练参数上是线性的，预测器精确到数值精度；对于非线性适配器/全微调，它是局部NTK近似。同一表达式揭示遗忘集中在少量旧任务NTK本征模式上，并在冻结线性头下给出脆弱秩的Kronecker缩放规则。这些结果澄清了与先前NTK重叠理论的关系，解释了为什么参数空间正则化器可能遗漏输出空间干扰，并激发了一种有针对性的谱正则化器。
英文摘要：
Catastrophic forgetting in continual adaptation is usually studied through parameter drift, replay, or distillation, but these views do not identify which output-space directions are vulnerable. We give a function-space account in the NTK regime: new-task training induces old-task prediction drift through the cross-task kernel, yielding a closed-form predictor for the forgetting vector before any new-task gradient step. In frozen-backbone linear-head PEFT-CL, where the model is linear in the trainable parameters, the predictor is exact up to numerical precision; for nonlinear adapters/full fine-tuning, it is a local NTK approximation. The same expression reveals that forgetting concentrates in a small number of old-task NTK eigenmodes and under frozen linear heads gives a Kronecker scaling rule for the vulnerable rank. These results clarify the relation to prior NTK-overlap theory, explain why parameter-space regularizers can miss output-space interference, and motivate a targeted spectral regularizer.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
