针对标签噪声的鲁棒监督对比损失的一个包容性理论框架
An Inclusive Theoretical Framework of Robust Supervised Contrastive Loss against Label Noise
AI总结：
本文提出统一理论框架，首次推导任意对比损失的鲁棒条件，证明InfoNCE不鲁棒并提出对称InfoNCE（SymNCE），同时解释最近邻样本选择等已有技术，实验验证其抗标签噪声优势。
摘要：
从噪声标签中学习是机器学习中的一个关键挑战，对众多现实世界场景具有广泛影响。虽然监督对比学习近来已成为应对标签噪声的有力工具，但许多现有解决方案仍是启发式的，往往缺乏构建鲁棒监督对比损失的系统性理论基础。为填补这一空白，本文在成对对比范式下提出了一个统一的鲁棒损失理论框架。特别是，我们首次推导出任意对比损失的一般鲁棒条件，该条件可作为验证监督对比损失对标签噪声具有理论鲁棒性的准则。该理论表明，流行的InfoNCE损失实际上并不鲁棒，并据此启发我们开发了InfoNCE的鲁棒版本，称为对称InfoNCE（SymNCE）。此外，我们强调我们的理论是一个包容性框架，能够为先前鲁棒技术（如最近邻（NN）样本选择和鲁棒对比损失）提供解释。在基准数据集上的验证实验证明了SymNCE在面对标签噪声时的优越性。
英文摘要：
Learning from noisy labels is a critical challenge in machine learning, with vast implications for numerous real-world scenarios. While supervised contrastive learning has recently emerged as a powerful tool for navigating label noise, many existing solutions remain heuristic, often devoid of a systematic theoretical foundation for crafting robust supervised contrastive losses. To address the gap, in this paper, we propose a unified theoretical framework for robust losses under the pairwise contrastive paradigm. In particular, we for the first time derive a general robust condition for arbitrary contrastive losses, which serves as a criterion to verify the theoretical robustness of a supervised contrastive loss against label noise. The theory indicates that the popular InfoNCE loss is in fact non-robust, and accordingly inspires us to develop a robust version of InfoNCE, termed Symmetric InfoNCE (SymNCE). Moreover, we highlight that our theory is an inclusive framework that provides explanations to prior robust techniques such as nearest-neighbor (NN) sample selection and robust contrastive loss. Validation experiments on benchmark datasets demonstrate the superiority of SymNCE against label noise.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
