按设计实现神经崩溃：在超球面上学习类别原型
Neural Collapse by Design: Learning Class Prototypes on the Hypersphere
AI总结：
本文研究了监督分类的理论最优解神经崩溃（NC），指出交叉熵（CE）和监督对比学习（SCL）两种主流范式在实践中无法达到该最优解。作者提出通过在超球面上对比原型的方法，改进了CE和SCL，从而在多个基准测试中实现了更接近NC的性能。
摘要：
监督分类有一个理论最优解，即神经崩溃（NC），然而其两种主导范式在实践中都无法达到这一最优。交叉熵（CE）保留了径向自由度，导致收敛到退化几何结构，而监督对比学习（SCL）在预训练阶段驱动特征向NC靠近，但在后续的线性探测阶段丢弃了这一结构。我们证明这两种范式实际上是同一种方法的不同表现，即在单位超球面上对比原型。缩小差距需要在各自失败点进行修正。从CE侧，我们提出NTCE和NONL两种归一化损失，将对比优化缺失的成分引入分类器学习：大有效负样本集和解耦的对齐和均匀性项。从SCL侧，我们证明SCL的目标在训练过程中已经优化了原理分类器，其权重是类别均值嵌入，使线性探测变得冗余且有害。实验表明，在四个基准测试（包括ImageNet-1K）中，NTCE和NONL在准确率上超过了CE，接近NC（≥95%），并在不到7.5%的迭代次数中在4/5个指标上匹配CE的收敛NC，而SCL在固定原型的情况下无需线性探测阶段即可达到。学习的几何结构在迁移学习中带来了+5.5%的平均相对改进，严重类别不平衡下可达+8.7%，并且在ImageNet-C上提高了对损坏的鲁棒性。本文将监督学习重新定义为在超球面上的原型学习，通过设计达到NC。
英文摘要：
Supervised classification has a theoretical optimum, Neural Collapse (NC), yet neither of its two dominant paradigms reaches it in practice. Cross entropy (CE) leaves radial degrees of freedom unconstrained and converges to a degenerate geometry, while supervised contrastive learning (SCL) drives features toward NC during pretraining but discards this structure in a post hoc linear probing phase. We show that both paradigms are different appearances of the same method that contrasts prototypes on the unit hypersphere, and that closing the gap requires fixing each at its point of failure. From the CE side, we propose NTCE and NONL, two normalized losses that import contrastive optimization's missing ingredients into classifier learning: a large effective negative set and decoupled alignment and uniformity terms. From the SCL side, we prove that SCL's objective already optimizes throughout training for a principled classifier whose weights are the class mean embeddings, making linear probing both redundant and harmful. Empirically, on four benchmarks including ImageNet-1K, NTCE and NONL surpass CE accuracy, closely approximate NC ($\geq 95\%$), and match CE's converged NC on 4/5 metrics in under $7.5\%$ of its iterations, while SCL with fixed prototypes matches linear probing without the hours-long classifier training phase. The learned geometry yields $+5.5\%$ mean relative improvement in transfer learning, up to $+8.7\%$ under severe class imbalance, and improved robustness to corruptions on ImageNet-C. Our work recasts supervised learning as prototype learning on the hypersphere, with NC reached by design.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
