arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2207.12876cs.LG
收藏

不变学习的重复环境推理

Repeated Environment Inference for Invariant Learning

Johns Hopkins University(约翰斯·霍普金斯大学)

Aayush Mishra, Anqi Liu

摘要:

研究了环境标签未知时的不变学习问题。我们专注于不变表示的概念时,贝叶斯最优条件标签分布是相同的,在不同的环境。以前的工作进行环境推理(EI)最大化的惩罚项从不变风险最小化(EI)框架。EI步骤使用关注伪相关性的参考模型,以有效地达到良好的环境分区。然而,如何找到这样一个参考模型尚不清楚。在这项工作中,我们建议重复EI过程,并在由前一个EI步骤推断的\textit{majority}环境上重新训练ERM模型。在温和的假设下,我们发现这种迭代过程有助于学习比单步的更好地捕捉虚假相关的表示。这导致更好的环境推理和更好的不变学习。我们表明,这种方法在合成和真实世界的数据集上都优于基线。

英文摘要:

We study the problem of invariant learning when the environment labels are unknown. We focus on the invariant representation notion when the Bayes optimal conditional label distribution is the same across different environments. Previous work conducts Environment Inference (EI) by maximizing the penalty term from Invariant Risk Minimization (IRM) framework. The EI step uses a reference model which focuses on spurious correlations to efficiently reach a good environment partition. However, it is not clear how to find such a reference model. In this work, we propose to repeat the EI process and retrain an ERM model on the \textit{majority} environment inferred by the previous EI step. Under mild assumptions, we find that this iterative process helps learn a representation capturing the spurious correlation better than the single step. This results in better Environment Inference and better Invariant Learning. We show that this method outperforms baselines on both synthetic and real-world datasets.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑