自监督对比学习近似于监督对比学习
Self-Supervised Contrastive Learning is Approximately Supervised Contrastive Learning
AI总结：
该研究填补自监督对比学习（CL）理论空白，证明其隐式近似仅负样本监督对比损失（NSCL），且两者差距随类别数增加消失，还刻画NSCL几何结构并提出少样本误差边界，实证验证理论发现。
摘要：
尽管自监督对比学习（CL）在实证中取得了成功，但其理论基础尚未完全确立。在这项工作中，我们通过证明标准CL目标隐式近似于一种监督变体（我们称之为仅负样本监督对比损失，NSCL）来填补这一空白，该变体排除了同类对比。我们证明，在与标签无关且与架构无关的边界条件下，CL和NSCL损失之间的差距会随着语义类别数量的增加而消失。我们刻画了NSCL损失全局极小值的几何结构：学习到的表示表现出增强崩溃、类内崩溃，且类中心形成单纯形等角紧框架。我们进一步引入了线性探测少样本误差的新边界，该边界取决于特征变异性的两个度量——类内离散度和类中心连线方向的变异度。我们表明，方向变异主导了该边界，且类内离散度的影响会随着标记样本数量的增加而减弱。这些特性使CL和NSCL训练的表示能够通过简单的线性探测支持准确的少样本标签恢复。最后，我们通过实证验证了理论发现：CL和NSCL损失之间的差距以$\boldsymbol{\frac{1}{\text{类别数量}}}$的速率衰减；两种损失高度相关；最小化CL损失会隐式使NSCL损失接近直接最小化所能达到的值；所提出的少样本误差边界在实践中对探测性能提供了紧密的估计。本文的代码和项目页面可在[code, project page]获取。
英文摘要：
Despite its empirical success, the theoretical foundations of self-supervised contrastive learning (CL) are not yet fully established. In this work, we address this gap by showing that standard CL objectives implicitly approximate a supervised variant we call the negatives-only supervised contrastive loss (NSCL), which excludes same-class contrasts. We prove that the gap between the CL and NSCL losses vanishes as the number of semantic classes increases, under a bound that is both label-agnostic and architecture-independent. We characterize the geometric structure of the global minimizers of the NSCL loss: the learned representations exhibit augmentation collapse, within-class collapse, and class centers that form a simplex equiangular tight frame. We further introduce a new bound on the few-shot error of linear-probing. This bound depends on two measures of feature variability--within-class dispersion and variation along the line between class centers. We show that directional variation dominates the bound and that the within-class dispersion's effect diminishes as the number of labeled samples increases. These properties enable CL and NSCL-trained representations to support accurate few-shot label recovery using simple linear probes. Finally, we empirically validate our theoretical findings: the gap between CL and NSCL losses decays at a rate of $\mathcal{O}(\frac{1}{\#\text{classes}})$; the two losses are highly correlated; minimizing the CL loss implicitly brings the NSCL loss close to the value achieved by direct minimization; and the proposed few-shot error bound provides a tight estimate of probing performance in practice. The code and project page of the paper are available at [\href{https://github.com/DLFundamentals/understanding-ssl}{code}, \href{https://dlfundamentals.github.io/ssl-is-approximately-sl/}{project page}].
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
