arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2107.00520cs.LGstat.ML
收藏

存在干扰诱导伪相关时的分布外泛化

Out-of-distribution Generalization in the Presence of Nuisance-Induced Spurious Correlations

New York University(纽约大学);Langone Health(朗格尼医学中心)

Aahlad Puli, Lily H. Zhang, Eric K. Oermann, Rajesh Ranganath

摘要:

在许多预测问题中,虚假的相关性是由标签和一个讨厌的变量之间的关系变化引起的,这个讨厌的变量也与协变量相关。例如,在对自然图像中的动物进行分类时,作为干扰的背景可以预测动物的类型。这种滋扰-标签关系并不总是成立的,并且在一种这样的关系下训练的模型的性能在具有不同滋扰-标签关系的数据上可能很差。为了建立无论滋扰标签关系如何都能表现良好的预测模型,我们开发了滋扰随机蒸馏(Nuisance-Randomized Distillation,简称NZD)。我们介绍了滋扰随机分布,滋扰和标签是独立的分布。在这种分布下,我们定义了一组表示,使条件的任何成员,滋扰和标签保持独立。我们证明了在这个集合中的表示总是比机会表现得更好,而在这个集合之外的表示可能不会。CNOD发现一个表示,从这个集合中,是最有用的滋扰随机分布下的标签,我们证明,这种表示达到最高的性能,无论滋扰标签的关系。我们评估了几个任务,包括胸部X射线分类,其中,使用非肺部补丁作为滋扰,CNORD产生的模型,预测肺炎下强虚假的相关性。

英文摘要:

In many prediction problems, spurious correlations are induced by a changing relationship between the label and a nuisance variable that is also correlated with the covariates. For example, in classifying animals in natural images, the background, which is a nuisance, can predict the type of animal. This nuisance-label relationship does not always hold, and the performance of a model trained under one such relationship may be poor on data with a different nuisance-label relationship. To build predictive models that perform well regardless of the nuisance-label relationship, we develop Nuisance-Randomized Distillation (NURD). We introduce the nuisance-randomized distribution, a distribution where the nuisance and the label are independent. Under this distribution, we define the set of representations such that conditioning on any member, the nuisance and the label remain independent. We prove that the representations in this set always perform better than chance, while representations outside of this set may not. NURD finds a representation from this set that is most informative of the label under the nuisance-randomized distribution, and we prove that this representation achieves the highest performance regardless of the nuisance-label relationship. We evaluate NURD on several tasks including chest X-ray classification where, using non-lung patches as the nuisance, NURD produces models that predict pneumonia under strong spurious correlations.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑