发表机构
BRAC University; Iowa State University（BRAC大学; 爱荷华州立大学）
机构由 AI 辅助整理，请以论文原文为准。AI 中文总结
本文指出一步曲率探针不足以评估持续学习，提出终端零空间校正方法，通过保留拟合容量和端点曲率分解终端代价，实验验证其能有效降低旧任务遗忘。
AI 中文摘要
一步曲率探针评估的是初始方向，而持续学习者的评判标准是在达到可比的新任务拟合之后。在过参数化线性化中，投影梯度下降收敛到 $\Delta_P=PJ^\top(JPJ^\top)^{-1}r$，其平方位移膨胀恰好是保留拟合容量 $c_P(r)$ 的倒数。更一般地，终端旧任务二次比率分解为 $1/[c_P(r)G_{\rm end}(P,r)]$，其中 $G_{\rm end}$ 比较沿端点方向的曲率。在秩为一的情形下，$G_{\rm end}$ 等于一步探针增益；对于多输出，两种增益可能不同。终端二次式还分离为曲率最优的拟合下限和依赖算法的零空间过剩，这促使进行终端零空间校正，该校正在其雅可比批次上保留线性化新任务输出。受控检查验证了局部二次式，并表明投影可以大幅降低匹配范数曲率，同时几乎不改变终端遗忘。在常见阈值配置中，投影在73/99个匹配对中产生更大的有符号旧任务损失变化，保留拟合容量随秩下降，且固定秩比较即使在探针增益近似匹配时也能区分终端遗忘。在Permuted MNIST和Split CIFAR-100上，终端校正使166/180个方法-数据集-种子对在有符号旧任务损失上降低，采用全种子意向校正分析；由于接受和结果报告使用相同的留出分割，该结果是描述性的且以测试为条件。总体而言，终端代价共同取决于保留拟合容量、端点方向曲率和优化所选择的零空间分量。
英文摘要
A one-step curvature probe evaluates an initial direction, whereas continual learners are judged after reaching comparable new-task fit. In an overparameterized linearization, projected gradient descent converges to $Δ_P=PJ^\top(JPJ^\top)^{-1}r$, and its squared-displacement inflation is exactly the reciprocal of the retained fitting capacity $c_P(r)$. More generally, the terminal old-task quadratic ratio factorizes as $1/[c_P(r)G_{\rm end}(P,r)]$, where $G_{\rm end}$ compares curvature along endpoint directions. In the rank-one case, $G_{\rm end}$ equals the one-step probe gain; for multiple outputs, the two gains can differ. The terminal quadratic also separates into a curvature-optimal fitting floor and an algorithm-dependent null-space excess, motivating terminal null-space correction, which preserves linearized new-task outputs on its Jacobian batch. Controlled checks validate the local quadratic and show that projection can greatly reduce matched-norm curvature while barely changing terminal forgetting. Across common-threshold configurations, projection yields the larger signed old-task loss change in 73/99 matched pairs, retained fitting capacity falls with rank, and fixed-rank comparisons separate terminal forgetting even when probe gain is approximately matched. On Permuted MNIST and Split CIFAR-100, terminal correction decreases signed old-task loss in 166/180 method--dataset--seed pairs under the all-seed intention-to-correct analysis; because acceptance and outcome reporting use the same held-out split, this result is descriptive and test-conditioned. Overall, terminal cost depends jointly on retained fitting capacity, endpoint-direction curvature, and the null-space component selected by optimization.
