基于重放的持续学习中始终开启的簇间排斥的失效模式
Failure Modes of Always-On Inter-Cluster Repulsion in Replay-Based Continual Learning
AI总结：
研究基于重放的持续学习中簇感知重放（CAR）里始终开启的簇间排斥的失效模式，结合类平衡重放记忆与排斥项，通过实验得出其负面结果，表明几何分离与重放非自动互补，激发边际门控目标。
摘要：
特征空间目标通常被添加到基于重放的持续学习系统中，期望更好的几何分离能提高保留率。我们研究了一种初步形式的簇感知重放（CAR），它将类平衡重放记忆与始终激活的簇间排斥项（ICF）相结合。在具有ResNet-18骨干的五任务分割CIFAR-10上，在六值灵敏度扫描中观察到的最高平均最终平均准确率在三个种子上达到$22.5\pm1.4\%$，而仅重放为$23.1\pm2.5\%$。没有重放的ICF仅达到$19.2\pm0.1\%$。所有测试的排斥权重产生的最终准确率在$20.1\%$和$22.5\%$之间，详细配置显示平均遗忘率为$89.2\pm1.5$个百分点。仪器化重新运行表明，在交叉熵接近零后，加权排斥贡献仍接近$-0.13$，因此总目标变为负数，而旧任务准确率崩溃。重要的是，归一化距离目标在数学上是有界的；因此，失败更好地描述为非饱和、始终开启的排斥，而不是无界损失。这些负面结果表明，几何分离并非自动与重放互补，并激发了边际门控目标，其梯度在达到足够分离后停用。
英文摘要：
Feature-space objectives are often added to replay-based continual learning systems with the expectation that better geometric separation will improve retention. We study a preliminary form of Cluster-Aware Replay (CAR) that combines a class-balanced replay memory with an always-active inter-cluster repulsion term (ICF). On five-task Split CIFAR-10 with a ResNet-18 backbone, the highest observed mean in a six-value sensitivity sweep reaches $22.5\pm1.4\%$ final average accuracy over three seeds, compared with $23.1\pm2.5\%$ for replay alone. ICF without replay reaches only $19.2\pm0.1\%$. All tested repulsion weights produce final accuracies between $20.1\%$ and $22.5\%$, and the detailed configuration exhibits $89.2\pm1.5$ percentage points of average forgetting. An instrumented rerun shows that the weighted repulsion contribution remains near $-0.13$ after cross-entropy has fallen close to zero, so the total objective becomes negative while old-task accuracy collapses. Importantly, the normalized distance objective is mathematically bounded; the failure is therefore better described as non-saturating, always-on repulsion rather than an unbounded loss. These negative results show that geometric separation is not automatically complementary to replay and motivate margin-gated objectives whose gradients deactivate once sufficient separation has been reached.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
