无偏监督对比学习
Unbiased Supervised Contrastive Learning
AI总结：
针对数据集偏差导致对比学习失效的问题，提出基于间隔的理论框架，推导epsilon-SupInfoNCE损失与FairKL去偏正则项，在多视觉数据集上实现SOTA去偏性能。
摘要：
许多数据集存在偏差，即它们包含仅在数据集中与目标类别高度相关、但在数据真实底层分布中并无此关联的易学习特征。正因如此，从有偏数据中学习无偏模型在近年成为极具相关性的研究主题。本研究致力于解决学习对偏差具备鲁棒性的表征这一问题。我们首先提出一个基于间隔的理论框架，该框架可帮助我们阐明近期的对比损失（如InfoNCE、SupCon等）在处理有偏数据时失效的原因。基于此，我们推导出一种监督对比损失的新公式epsilon-SupInfoNCE，它能更精准地控制正负样本之间的最小距离。此外，依托我们的理论框架，我们还提出了FairKL这一新型去偏差正则化损失，即便在存在极端偏差的数据上也能表现良好。我们在CIFAR10、CIFAR100和ImageNet等标准视觉数据集上验证了所提出的损失函数，并评估了结合epsilon-SupInfoNCE的FairKL的去偏差能力，其在多个有偏数据集（包括真实场景下的偏差实例）上达到了当前最优性能。
英文摘要：
Many datasets are biased, namely they contain easy-to-learn features that are highly correlated with the target class only in the dataset but not in the true underlying distribution of the data. For this reason, learning unbiased models from biased data has become a very relevant research topic in the last years. In this work, we tackle the problem of learning representations that are robust to biases. We first present a margin-based theoretical framework that allows us to clarify why recent contrastive losses (InfoNCE, SupCon, etc.) can fail when dealing with biased data. Based on that, we derive a novel formulation of the supervised contrastive loss (epsilon-SupInfoNCE), providing more accurate control of the minimal distance between positive and negative samples. Furthermore, thanks to our theoretical framework, we also propose FairKL, a new debiasing regularization loss, that works well even with extremely biased data. We validate the proposed losses on standard vision datasets including CIFAR10, CIFAR100, and ImageNet, and we assess the debiasing capability of FairKL with epsilon-SupInfoNCE, reaching state-of-the-art performance on a number of biased datasets, including real instances of biases in the wild.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
