通过logit校正避免虚假相关
Avoiding spurious correlations via logit correction
摘要：
实证研究表明，用经验风险最小化（ERM）训练的机器学习模型通常依赖于可能与类别标签虚假相关的属性。这样的模型通常导致在对缺乏这种相关性的数据进行推断期间性能较差。在这项工作中，我们明确考虑的情况下，潜在的虚假相关性存在于大多数的训练数据。与现有的方法相比，这些方法使用ERM模型输出来检测没有虚假相关性的样本，并对这些样本进行逐层加权或上采样，我们提出了logit校正（LC）损失，这是对softmax交叉熵损失的一种简单而有效的改进，以校正样本logit。我们证明了最小化LC损失相当于最大化组平衡精度，因此所提出的LC可以减轻虚假相关的负面影响。我们广泛的实验结果进一步表明，所提出的LC损失优于国家的最先进的解决方案，在多个流行的基准的大幅度，平均5.5%的绝对改善，没有访问虚假的属性标签。LC也与使用属性标签的Oracle方法竞争。代码可在https://github.com/shengliu66/LC上获得。
英文摘要：
Empirical studies suggest that machine learning models trained with empirical risk minimization (ERM) often rely on attributes that may be spuriously correlated with the class labels. Such models typically lead to poor performance during inference for data lacking such correlations. In this work, we explicitly consider a situation where potential spurious correlations are present in the majority of training data. In contrast with existing approaches, which use the ERM model outputs to detect the samples without spurious correlations and either heuristically upweight or upsample those samples, we propose the logit correction (LC) loss, a simple yet effective improvement on the softmax cross-entropy loss, to correct the sample logit. We demonstrate that minimizing the LC loss is equivalent to maximizing the group-balanced accuracy, so the proposed LC could mitigate the negative impacts of spurious correlations. Our extensive experimental results further reveal that the proposed LC loss outperforms state-of-the-art solutions on multiple popular benchmarks by a large margin, an average 5.5\% absolute improvement, without access to spurious attribute labels. LC is also competitive with oracle methods that make use of the attribute labels. Code is available at https://github.com/shengliu66/LC.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
