基于异构相似度的监督对比学习应对分布偏移
Supervised Contrastive Learning with Heterogeneous Similarity for Distribution Shifts
AI总结：
提出基于异构相似度的监督对比学习正则化方法，通过在对比损失中引入可变参数构建边际效应，有效缓解过拟合并提升模型在分布偏移下的泛化能力。
摘要：
分布偏移是指训练和测试之间数据分布发生变化的问题，这会显著降低模型在现实世界部署时的性能。最近的研究表明，性能下降的一个原因是一类过拟合，而适当的正则化可以缓解这种下降，特别是在使用神经网络等高度代表性的模型时。在本文中，我们提出了一种使用监督对比学习的新正则化方法，以防止这种过拟合并训练在分布偏移下不降低性能的模型。我们将对比损失中的余弦相似度扩展为更一般的相似度度量，并建议在度量中使用不同参数来比较样本与正例或负例，分析表明这在对比损失中起到了一种边际的作用。在模拟分布偏移的基准数据集（包括子种群偏移和领域泛化））上的实验表明，所提出的方法优于现有的正则化方法。
英文摘要：
Distribution shifts are problems where the distribution of data changes between training and testing, which can significantly degrade the performance of a model deployed in the real world. Recent studies suggest that one reason for the degradation is a type of overfitting, and that proper regularization can mitigate the degradation, especially when using highly representative models such as neural networks. In this paper, we propose a new regularization using the supervised contrastive learning to prevent such overfitting and to train models that do not degrade their performance under the distribution shifts. We extend the cosine similarity in contrastive loss to a more general similarity measure and propose to use different parameters in the measure when comparing a sample to a positive or negative example, which is analytically shown to act as a kind of margin in contrastive loss. Experiments on benchmark datasets that emulate distribution shifts, including subpopulation shift and domain generalization, demonstrate the advantage of the proposed method over existing regularization methods.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
