用于长尾分布的频谱感知分析类增量学习
Spectral-Aware Analytic Class-Incremental Learning for Long-Tailed Distributions
AI总结：
研究针对长尾分布的类增量学习问题，提出几何频谱校正（GSR）框架，将其视为频谱正则化问题，构造频谱扰动矩阵有选择地增大尾部类坍缩特征值，实验表明该方法在计算效率和鲁棒泛化间取得更好权衡，达新最优水平。
AI中文摘要：
分析持续学习（ACL）为基于梯度的方法提供了一种计算高效的替代方案。最近的ACL方法基于递归最小二乘法（RLS），与其他方法相比取得了最优结果。然而，在具有长尾分布的类增量学习场景中，它们表现不佳。虽然自相关（Gram）矩阵的病态是RLS的已知局限性，但我们证明类不平衡会将此问题加剧为一种独特的频谱病态：“尾部”类会出现严重的频谱坍缩，使其子空间在数值上与噪声无法区分。标准岭回归（$L_2$）无法有效解决此问题，因为它应用的是各向同性正则化，这种均匀惩罚不足以稳定尾部而不使头部过度收缩。为了解决这个问题，我们提出了几何频谱校正（GSR），这是一个理论基础框架，将长尾学习视为频谱正则化问题。与均匀惩罚所有特征值的标准各向同性正则化（岭回归）不同，GSR充当各向异性频谱滤波器，有选择地增大尾部类坍缩的特征值。我们构造了一个结构化、数据依赖的频谱扰动矩阵$\Delta$，有选择地增大Gram矩阵坍缩的尾部特征方向。理论分析证明GSR保证了Gram矩阵的稳定秩得到改善，确保了数值稳定性。大量实验表明，GSR在分析类增量学习方面建立了新的最优水平，在长尾设置下的计算效率和鲁棒泛化之间提供了更好的权衡。
英文摘要：
Analytic Continual Learning (ACL) offers a computationally efficient alternative to gradient-based approaches. Recent ACL methods are based on Recursive Least Squares (RLS) and have achieved the state-of-the-art results compared to other alternatives. However, they falter significantly in Class-Incremental Learning scenarios characterized by Long-Tailed distributions. While the ill-conditioning of the autocorrelation (Gram) matrix is a known limitation of RLS, we demonstrate that class imbalance exacerbates this issue into a distinct spectral pathology: "tail" classes suffer from severe spectral collapse, rendering their subspaces numerically indistinguishable from noise. Standard Ridge Regression ($L_2$) fails to address this effectively as it applies isotropic regularization - a uniform penalty that is insufficient to stabilize the tail without over-shrinking the head. To address this, we propose Geometry-Spectral Rectification (GSR), a theoretically grounded framework that treats long-tailed learning as a spectral regularization problem. Unlike standard isotropic regularization (Ridge) which uniformly penalizes all eigenvalues, GSR acts as an anisotropic spectral filter, selectively inflating the collapsed eigenvalues of tail classes. We construct a structured, data-dependent spectral perturbation matrix $Δ$ that selectively inflates collapsed tail eigen-directions of the Gram matrix. Theoretical analysis proves that GSR guarantees an improved stable rank for the Gram matrix, ensuring numerical stability. Extensive experiments show that GSR establishes a new state-of-the-art for analytic CIL, offering a superior trade-off between computational efficiency and robust generalization in long-tailed settings.
