arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2203.01517cs.LG
收藏

Correct-N-Contrast:一种提升对虚假相关性鲁棒性的对比学习方法

Correct-N-Contrast: A Contrastive Approach for Improving Robustness to Spurious Correlations

Stanford University(斯坦福大学);Northeastern University(东北大学)

Michael Zhang, Nimit S. Sohoni, Hongyang R. Zhang, Chelsea Finn, Christopher Ré

AI总结:

针对虚假相关性导致机器学习模型最差组性能差、且无虚假属性标签时难优化的问题,提出对比学习方法CNC,借助ERM模型识别同类异虚假特征样本并做对比学习,大幅降低对齐损失,刷新无属性标签下最差组准确率最优水平。

摘要:

虚假相关性是鲁棒机器学习面临的一大核心挑战。采用经验风险最小化(ERM)训练的模型可能会习得对类别标签与虚假属性之间相关性的依赖,导致在不具备这类相关性的数据组上表现不佳。当无法获取虚假属性标签时,解决这一问题的难度会进一步提升。为了在无需属性标签训练的前提下提升含虚假相关性数据上的最差组性能,我们提出Correct-N-Contrast(CNC)—— 一种可直接学习对虚假相关性具备鲁棒性的表征的对比学习方法。由于ERM模型可作为性能良好的虚假属性预测器,CNC的工作流程分为两步:(1)利用已训练ERM模型的输出,识别类别相同但虚假特征存在差异的样本;(2)通过对比学习训练鲁棒模型,为同类别样本学习相似的表征。为支撑CNC的有效性,我们建立了最差组误差与CNC旨在最小化的表征对齐损失之间的全新关联。我们通过实验观察到,最差组误差与对齐损失的变化高度一致,并证明了某一类别的对齐损失可为该类别最差组误差与平均误差的差值提供上界。在主流基准测试上,CNC大幅降低了对齐损失,且最差组准确率达到当前最优水平,平均绝对提升幅度为3.6%。CNC的性能也与需要组标签的先知方法具备竞争力。

英文摘要:

Spurious correlations pose a major challenge for robust machine learning. Models trained with empirical risk minimization (ERM) may learn to rely on correlations between class labels and spurious attributes, leading to poor performance on data groups without these correlations. This is particularly challenging to address when spurious attribute labels are unavailable. To improve worst-group performance on spuriously correlated data without training attribute labels, we propose Correct-N-Contrast (CNC), a contrastive approach to directly learn representations robust to spurious correlations. As ERM models can be good spurious attribute predictors, CNC works by (1) using a trained ERM model's outputs to identify samples with the same class but dissimilar spurious features, and (2) training a robust model with contrastive learning to learn similar representations for same-class samples. To support CNC, we introduce new connections between worst-group error and a representation alignment loss that CNC aims to minimize. We empirically observe that worst-group error closely tracks with alignment loss, and prove that the alignment loss over a class helps upper-bound the class's worst-group vs. average error gap. On popular benchmarks, CNC reduces alignment loss drastically, and achieves state-of-the-art worst-group accuracy by 3.6% average absolute lift. CNC is also competitive with oracle methods that require group labels.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑