发表机构
University of Sheffield(谢菲尔德大学)
机构由 AI 辅助整理,请以论文原文为准。AI 中文总结
SPARCL 是一种谱分区解析式持续学习方法,通过分解自相关为高能核心与残差,冻结旧类别分类器核心,更新残差块,在多数据集上缩小了经典解析式学习器与强表示匹配器的性能差距,且与 Fly-CL 等方法互补。
AI 中文摘要
解析式持续学习已成为基于梯度的类增量学习的一种强大的无示例替代方案,因为它用闭式岭更新取代了迭代优化。然而,以随机梯度覆盖为核心的常见遗忘机制并不能解释,为何即便有精确的递归求解器,解析式方法仍会在旧类别上发生漂移。我们将罪魁祸首确定为谱干扰:适用于所有任务的联合岭分类器共享逆自相关算子 $(R+\lambda I)^{-1}$,因此加载到旧主导特征方向上的新任务样本会稀释谱并扰动旧类别 logits,即便从未重新访问旧标签。基于这一观点,我们提出 SPARCL,一种谱分区解析式持续学习器,它将运行自相关分解为高能核心和残差补集,冻结核心子空间中的旧类别分类器组件,仅通过递归最小二乘(可选残差随机投影扩展)更新残差块。这产生了一种简单的闭式更新,对旧 logits 的核心贡献具有可证明的不变性保证。在冻结 ViT-B/16 协议下的 CIFAR-100、CUB-200、ImageNet-R 和 ImageNet-A 上,SPARCL 缩小了经典解析式学习器与强表示匹配器之间的大部分差距,同时与稀疏特征去相关方法(如 Fly-CL)互补。
英文摘要
Analytic continual learning has emerged as a strong exemplar-free alternative to gradient-based class-incremental learning because it replaces iterative optimization with closed-form ridge updates. Yet the usual forgetting narrative, centered on stochastic gradient overwriting, does not explain why analytic methods still drift on old classes despite exact recursive solvers. We identify the culprit as spectral interference: the joint ridge classifier for all tasks shares the inverse autocorrelation operator $(R+λI)^{-1}$, so incoming task samples that load onto old dominant eigendirections dilute the spectrum and perturb old-class logits even when old labels are never revisited. Based on this view, we propose SPARCL, a spectral partitioned analytic continual learner that decomposes the running autocorrelation into a high-energy core and a residual complement, freezes old-class classifier components in the core subspace, and updates only the residual block through recursive least squares with an optional residual random-projection expansion. This yields a simple closed-form update with a provable invariance guarantee for the core contribution of old logits. Across CIFAR-100, CUB-200, ImageNet-R, and ImageNet-A under a frozen ViT-B/16 protocol, SPARCL closes most of the gap from classical analytic learners to strong representation matchers, while remaining complementary to sparse feature-decorrelation approaches such as Fly-CL.