基于重平衡对比损失的长尾学习
Long-Tail Learning with Rebalanced Contrastive Loss
AI总结：
针对传统对比学习在长尾场景下偏向头部类别的问题，提出重平衡对比学习RCL，从特征空间平衡、类内紧凑性、正则化三方面优化，在BCL框架和独立使用时均取得SOTA性能。
摘要：
近期有研究提出将监督对比损失整合到基于交叉熵的分类中，作为解决长尾学习问题的方案。但当类别不平衡率较高时，需要调整监督对比损失以支撑尾部类别，因为传统对比学习默认偏向头部类别。为此，我们提出重平衡对比学习（Rebalanced Contrastive Learning，RCL），这是一种提升长尾分类准确率的高效方法，主要从三个方面着手：1. 特征空间平衡性——在所有类别间均等划分特征空间；2. 类内紧凑性——缩小同类嵌入向量之间的距离；3. 正则化——为尾部类别施加更大的间隔以减少过拟合。RCL将基于类别频率的SoftMax损失平衡方法应用于监督对比学习损失，并利用输入到对比学习损失的标量乘特征来增强紧凑性。我们在具备当前最优（SOTA）性能的Balanced Contrastive Learning（BCL，平衡对比学习）框架上实现了RCL。我们在三个基准数据集上的实验表明，RCL为BCL框架带来了更丰富的学习嵌入，且提升了top-1平衡准确率。我们进一步证明，RCL作为独立损失函数使用时，其性能也能达到当前最优水平的准确率。
英文摘要：
Integrating supervised contrastive loss to cross entropy-based communication has recently been proposed as a solution to address the long-tail learning problem. However, when the class imbalance ratio is high, it requires adjusting the supervised contrastive loss to support the tail classes, as the conventional contrastive learning is biased towards head classes by default. To this end, we present Rebalanced Contrastive Learning (RCL), an efficient means to increase the long tail classification accuracy by addressing three main aspects: 1. Feature space balancedness - Equal division of the feature space among all the classes, 2. Intra-Class compactness - Reducing the distance between same-class embeddings, 3. Regularization - Enforcing larger margins for tail classes to reduce overfitting. RCL adopts class frequency-based SoftMax loss balancing to supervised contrastive learning loss and exploits scalar multiplied features fed to the contrastive learning loss to enforce compactness. We implement RCL on the Balanced Contrastive Learning (BCL) Framework, which has the SOTA performance. Our experiments on three benchmark datasets demonstrate the richness of the learnt embeddings and increased top-1 balanced accuracy RCL provides to the BCL framework. We further demonstrate that the performance of RCL as a standalone loss also achieves state-of-the-art level accuracy.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
