通过监督目标理解自监督对比学习
Understanding Self-supervised Contrastive Learning through Supervised Objectives
AI总结：
本研究通过将自监督表示学习表述为监督学习目标的近似，推导出与InfoNCE相关的损失函数，引入原型表示偏差和平衡对比损失概念，从理论角度解释并改进了自监督对比学习算法。
摘要：
自监督表示学习取得了令人瞩目的经验性成功，但其理论理解仍然有限。在本研究中，我们通过将自监督表示学习表述为对监督表示学习目标的近似，提供了一个理论视角。基于这一表述，我们推导出一个与InfoNCE等流行对比损失密切相关的损失函数，揭示了其底层原理。我们的推导自然引入了原型表示偏差和平衡对比损失的概念，这有助于解释和改进自监督学习算法的行为。我们进一步展示了理论框架的组件如何对应于对比学习中的既有实践。最后，我们实证验证了平衡正负样本对交互的效果。所有理论证明在附录中提供，代码包含在补充材料中。
英文摘要：
Self-supervised representation learning has achieved impressive empirical success, yet its theoretical understanding remains limited. In this work, we provide a theoretical perspective by formulating self-supervised representation learning as an approximation to supervised representation learning objectives. Based on this formulation, we derive a loss function closely related to popular contrastive losses such as InfoNCE, offering insight into their underlying principles. Our derivation naturally introduces the concepts of prototype representation bias and a balanced contrastive loss, which help explain and improve the behavior of self-supervised learning algorithms. We further show how components of our theoretical framework correspond to established practices in contrastive learning. Finally, we empirically validate the effect of balancing positive and negative pair interactions. All theoretical proofs are provided in the appendix, and our code is included in the supplementary material.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
