arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2206.14534cs.LGstat.ML
收藏

什么时候群体不变学习能克服伪相关?

When Does Group Invariant Learning Survive Spurious Correlations?

Academy of Mathematics and Systems Science, Chinese Academy of Sciences(中国科学院数学与系统科学研究院);University of Chinese Academy of Sciences(中国科学院大学);Baidu Inc.(百度公司);Institute for AI Industry Research, Tsinghua University(清华大学智能产业研究院);Beijing Academy of Artificial Intelligence(北京智源人工智能研究院)

Yimeng Chen, Ruibin Xiong, Zhiming Ma, Yanyan Lan

摘要:

通过推断潜在的组中的训练数据,最近的作品引入不变学习的情况下,环境注释是不可用的。通常,经验表明,在多数/少数分裂下的学习组不变性在许多数据集上有效地改善了分布外泛化。然而,这些方法的学习不变机制缺乏理论保证。在本文中,我们揭示了现有的组不变学习方法在防止分类器依赖于训练集中的虚假相关性的不足。具体而言,我们提出了两个标准来判断这种充分性。理论和经验,我们表明,现有的方法可以违反这两个标准,从而无法概括到虚假的相关位移。基于此,本文设计了一种新的组不变学习方法,该方法通过统计独立性检验构造组,并根据组标签比例对样本进行重新加权以满足标准。在合成数据和真实数据上的实验表明,新方法在推广伪相关移位方面明显优于现有的组不变学习方法。

英文摘要:

By inferring latent groups in the training data, recent works introduce invariant learning to the case where environment annotations are unavailable. Typically, learning group invariance under a majority/minority split is empirically shown to be effective in improving out-of-distribution generalization on many datasets. However, theoretical guarantee for these methods on learning invariant mechanisms is lacking. In this paper, we reveal the insufficiency of existing group invariant learning methods in preventing classifiers from depending on spurious correlations in the training set. Specifically, we propose two criteria on judging such sufficiency. Theoretically and empirically, we show that existing methods can violate both criteria and thus fail in generalizing to spurious correlation shifts. Motivated by this, we design a new group invariant learning method, which constructs groups with statistical independence tests, and reweights samples by group label proportion to meet the criteria. Experiments on both synthetic and real data demonstrate that the new method significantly outperforms existing group invariant learning methods in generalizing to spurious correlation shifts.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑