头部坍缩，特征保持：为什么Replay需要大缓冲区
Heads collapse, features stay: Why Replay needs big buffers
AI总结：
研究揭示持续学习中特征空间与分类器层面遗忘的不对称性，提出通过扩展神经坍塌框架来解释Replay缓冲区大小对性能的影响，指出小缓冲区导致统计退化，需通过修正统计退化实现鲁棒性。
摘要：
持续学习中存在一个持续的悖论：即使神经网络的输出预测失败，它们仍保留线性可分的过去任务表示。我们正式将这种区别定义为深度（特征空间）与浅层（分类器层面）遗忘之间的差距。我们揭示了经验回放中的关键不对称性：虽然最小缓冲区能锚定特征几何并防止深度遗忘，但缓解浅层遗忘通常需要显著更大的缓冲容量。为此，我们扩展了神经坍塌框架到序列设置。我们把深度遗忘描述为几何漂向分布外子空间，并证明任何非零回放比例渐近保证线性可分性的保留。相反，我们发现小缓冲区引起的“强坍塌”导致协方差秩缺陷和类均值膨胀，使分类器无法感知真实总体边界。通过将持续学习与分布外检测统一，我们的工作挑战了对大缓冲区的普遍依赖，表明显式修正这些统计退化可解锁最小回放下的鲁棒性能。
英文摘要：
A persistent paradox in continual learning (CL) is that neural networks often retain linearly separable representations of past tasks even when their output predictions fail. We formalize this distinction as the gap between deep (feature-space) and shallow (classifier-level) forgetting. We reveal a critical asymmetry in Experience Replay: while minimal buffers successfully anchor feature geometry and prevent deep forgetting, mitigating shallow forgetting typically requires substantially larger buffer capacities. To explain this, we extend the Neural Collapse framework to the sequential setting. We characterize deep forgetting as a geometric drift toward out-of-distribution subspaces and prove that any non-zero replay fraction asymptotically guarantees the retention of linear separability. Conversely, we identify that the ``strong collapse'' induced by small buffers leads to rank-deficient covariances and inflated class means, effectively blinding the classifier to true population boundaries. By unifying CL with out-of-distribution detection, our work challenges the prevailing reliance on large buffers, suggesting that explicitly correcting these statistical artifacts could unlock robust performance with minimal replay.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
