监督对比学习中防止类别坍缩的理论框架
A Theoretical Framework for Preventing Class Collapse in Supervised Contrastive Learning
AI总结：
本文提出单纯形到单纯形嵌入模型（SSEM）作为理论框架，分析监督对比学习中超参数对表示的影响，并给出防止类别坍缩的实用指导，且通过合成与真实数据集实验验证了理论结论。
摘要：
监督对比学习（SupCL）已成为表示学习中的一种重要方法，同时利用监督损失和自监督损失。然而，在这两种损失之间取得最优平衡具有挑战性；若未能做到这一点，可能导致类别坍缩，从而降低同一类别中各个嵌入之间的区分度。在本文中，我们为 SupCL 提出了具有理论依据的指导原则，以防止学习到的表示发生类别坍缩。具体而言，我们引入了单纯形到单纯形嵌入模型（SSEM），这是一个理论框架，用于建模各种嵌入结构，包括所有使监督对比损失最小化的嵌入。通过 SSEM，我们分析了超参数如何影响学习到的表示，从而为超参数选择提供实用指导，以降低类别坍缩的风险。我们的理论发现得到了合成数据集和真实世界数据集上实证结果的支持。
英文摘要：
Supervised contrastive learning (SupCL) has emerged as a prominent approach in representation learning, leveraging both supervised and self-supervised losses. However, achieving an optimal balance between these losses is challenging; failing to do so can lead to class collapse, reducing discrimination among individual embeddings in the same class. In this paper, we present theoretically grounded guidelines for SupCL to prevent class collapse in learned representations. Specifically, we introduce the Simplex-to-Simplex Embedding Model (SSEM), a theoretical framework that models various embedding structures, including all embeddings that minimize the supervised contrastive loss. Through SSEM, we analyze how hyperparameters affect learned representations, offering practical guidelines for hyperparameter selection to mitigate the risk of class collapse. Our theoretical findings are supported by empirical results across synthetic and real-world datasets.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
