arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2007.06661cs.LGstat.ML
收藏

基于人工注释的伪相关鲁棒性研究

Robustness to Spurious Correlations via Human Annotations

Stanford University(斯坦福大学)

Megha Srivastava, Tatsunori Hashimoto, Percy Liang

摘要:

机器学习系统的可靠性严格假设特征和标签之间的关联在训练和测试分布之间保持相似。然而,不可测量的变量,如混杂因素,打破了这一假设--在训练时特征和标签之间有用的相关性在测试时可能变得无用甚至有害。例如,高度肥胖通常预示着心脏病,但这种关系可能不适用于吸烟者,他们通常肥胖率较低,心脏病发病率较高。我们提出了一个框架,使模型鲁棒虚假的相关性,利用人类的常识知识的因果关系。具体来说,我们使用人类注释来增强每个训练样本与一个潜在的不可测量的变量(即体重不足的心脏病患者可能是一个吸烟者),减少了问题的协变量偏移问题。然后,我们引入了一个新的分布鲁棒优化目标不可测变量(UV-DRO),以控制最坏情况下的损失超过可能的测试时间的变化。从经验上讲,我们表现出5-10%的数字识别任务混淆旋转,和1.5-5%的任务分析纽约警察局警察站混淆的位置。

英文摘要:

The reliability of machine learning systems critically assumes that the associations between features and labels remain similar between training and test distributions. However, unmeasured variables, such as confounders, break this assumption---useful correlations between features and labels at training time can become useless or even harmful at test time. For example, high obesity is generally predictive for heart disease, but this relation may not hold for smokers who generally have lower rates of obesity and higher rates of heart disease. We present a framework for making models robust to spurious correlations by leveraging humans' common sense knowledge of causality. Specifically, we use human annotation to augment each training example with a potential unmeasured variable (i.e. an underweight patient with heart disease may be a smoker), reducing the problem to a covariate shift problem. We then introduce a new distributionally robust optimization objective over unmeasured variables (UV-DRO) to control the worst-case loss over possible test-time shifts. Empirically, we show improvements of 5-10% on a digit recognition task confounded by rotation, and 1.5-5% on the task of analyzing NYPD Police Stops confounded by location.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑