一种用于自训练的统一对比损失
A Unified Contrastive Loss for Self-Training
AI总结：
针对半监督自训练多依赖交叉熵损失的问题，提出用统一对比损失替换所有交叉熵损失的通用框架，结合类别原型实现理论等价，在少标记数据集上提升性能，同时优化收敛速度、迁移能力与超参数稳定性。
摘要：
自训练方法已被证明能有效利用半监督学习中丰富的未标记数据，尤其是在标记数据稀缺的场景下。尽管这类方法大多依赖交叉熵损失函数（CE），但近期研究进展表明，监督对比损失函数（SupCon）效果更优。此外，无监督对比学习方法也已被证明能在无监督场景下捕捉高质量的数据表征。为在半监督场景下发挥这些优势，我们提出一个增强自训练方法的通用框架，该框架将所有CE损失替换为一种统一的对比损失。通过使用类别原型（即一组按类别划分的可训练参数），我们恢复了CE设置下的概率分布，并证明了二者在理论上的等价性。将我们的框架应用于主流自训练方法后，在标记数据数量有限的三个不同数据集上均实现了显著的性能提升。此外，我们还验证了该框架在收敛速度、迁移能力和超参数稳定性方面的进一步改进。代码可在https://github.com/AurelienGauffre/semisupcon/ 获取。
英文摘要：
Self-training methods have proven to be effective in exploiting abundant unlabeled data in semi-supervised learning, particularly when labeled data is scarce. While many of these approaches rely on a cross-entropy loss function (CE), recent advances have shown that the supervised contrastive loss function (SupCon) can be more effective. Additionally, unsupervised contrastive learning approaches have also been shown to capture high quality data representations in the unsupervised setting. To benefit from these advantages in a semi-supervised setting, we propose a general framework to enhance self-training methods, which replaces all instances of CE losses with a unique contrastive loss. By using class prototypes, which are a set of class-wise trainable parameters, we recover the probability distributions of the CE setting and show a theoretical equivalence with it. Our framework, when applied to popular self-training methods, results in significant performance improvements across three different datasets with a limited number of labeled data. Additionally, we demonstrate further improvements in convergence speed, transfer ability, and hyperparameter stability. The code is available at \url{https://github.com/AurelienGauffre/semisupcon/}.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
