广义监督对比学习
Generalized Supervised Contrastive Learning
AI总结：
本文提出广义监督对比损失，通过衡量标签相似性与潜在相似性间的交叉熵以充分利用标签分布，构建 GenSCL 框架并结合图像正则化与任意教师分类器，在 ImageNet、CIFAR10 和 CIFAR100 上取得新最优准确率。
摘要：
随着对比学习在自监督学习范式中近期取得的有前景的结果，监督对比学习已成功将这些对比方法扩展到监督语境中，在多个数据集上超越了交叉熵。然而，监督对比学习本质上以二元形式——正或负——利用标签信息，使用独热目标向量。这种结构难以适应将标签信息作为概率分布利用的方法，如 CutMix 和知识蒸馏。本文引入了一种广义监督对比损失，它衡量标签相似性与潜在相似性之间的交叉熵。这一概念通过充分利用标签分布并使各种现有技术能够适应现代神经网络的训练，增强了监督对比损失的能力。利用这种广义监督对比损失，我们构建了一个量身定制的框架：广义监督对比学习（GenSCL）。与现有对比学习框架相比，GenSCL 包含了额外的增强，包括先进的基于图像的正则化技术和一个任意教师分类器。当应用于结合 Momentum Contrast 技术的 ResNet50 时，GenSCL 在 ImageNet 上实现了 77.3% 的 top-1 准确率，较传统监督对比学习有 4.1% 的相对提升。此外，我们的方法在应用于 ResNet50 时，分别在 CIFAR10 和 CIFAR100 上建立了 98.2% 和 87.0% 的新最优准确率，标志着该架构报告的最高数据。
英文摘要：
With the recent promising results of contrastive learning in the self-supervised learning paradigm, supervised contrastive learning has successfully extended these contrastive approaches to supervised contexts, outperforming cross-entropy on various datasets. However, supervised contrastive learning inherently employs label information in a binary form--either positive or negative--using a one-hot target vector. This structure struggles to adapt to methods that exploit label information as a probability distribution, such as CutMix and knowledge distillation. In this paper, we introduce a generalized supervised contrastive loss, which measures cross-entropy between label similarity and latent similarity. This concept enhances the capabilities of supervised contrastive loss by fully utilizing the label distribution and enabling the adaptation of various existing techniques for training modern neural networks. Leveraging this generalized supervised contrastive loss, we construct a tailored framework: the Generalized Supervised Contrastive Learning (GenSCL). Compared to existing contrastive learning frameworks, GenSCL incorporates additional enhancements, including advanced image-based regularization techniques and an arbitrary teacher classifier. When applied to ResNet50 with the Momentum Contrast technique, GenSCL achieves a top-1 accuracy of 77.3% on ImageNet, a 4.1% relative improvement over traditional supervised contrastive learning. Moreover, our method establishes new state-of-the-art accuracies of 98.2% and 87.0% on CIFAR10 and CIFAR100 respectively when applied to ResNet50, marking the highest reported figures for this architecture.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
