arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2203.14237cs.LGcs.CV
收藏

基于因果关系的领域泛化表示学习

Causality Inspired Representation Learning for Domain Generalization

Beijing Institute of Technology(北京理工大学);Alibaba Group(阿里巴巴集团)

Fangrui Lv, Jian Liang, Shuang Li, Bin Zang, Chi Harold Liu, Ziteng Wang, Di Liu

摘要:

领域泛化本质上是一个分布外问题,旨在将从多个源领域学习到的知识泛化到一个未知的目标领域。主流是利用统计模型来建模数据和标签之间的依赖关系,旨在学习独立于领域的表示。然而,统计模型是对现实的肤浅描述,因为它们只需要对依赖性而不是内在的因果机制进行建模。当相关性随目标分布而变化时,统计模型可能无法推广。在这方面,我们引入了一个一般的结构因果模型来形式化DG问题。具体来说,我们假设每个输入都是由因果因素(其与标签的关系在各个领域都是不变的)和非因果因素(与类别无关)的混合构成的,并且只有前者导致分类判断。我们的目标是从输入中提取因果因素,然后重建不变的因果机制。然而,理论上的想法是远离实际的DG,因为所需的因果/非因果因素是不可观察的。我们强调,理想的因果因素应满足三个基本属性:分离的非因果的,共同独立的,因果关系足以分类。在此基础上,我们提出了一个因果启发表示学习(CIRL)算法,强制表示满足上述性质,然后使用它们来模拟因果因素,从而提高泛化能力。在几个广泛使用的数据集上的实验结果验证了该方法的有效性。

英文摘要:

Domain generalization (DG) is essentially an out-of-distribution problem, aiming to generalize the knowledge learned from multiple source domains to an unseen target domain. The mainstream is to leverage statistical models to model the dependence between data and labels, intending to learn representations independent of domain. Nevertheless, the statistical models are superficial descriptions of reality since they are only required to model dependence instead of the intrinsic causal mechanism. When the dependence changes with the target distribution, the statistic models may fail to generalize. In this regard, we introduce a general structural causal model to formalize the DG problem. Specifically, we assume that each input is constructed from a mix of causal factors (whose relationship with the label is invariant across domains) and non-causal factors (category-independent), and only the former cause the classification judgments. Our goal is to extract the causal factors from inputs and then reconstruct the invariant causal mechanisms. However, the theoretical idea is far from practical of DG since the required causal/non-causal factors are unobserved. We highlight that ideal causal factors should meet three basic properties: separated from the non-causal ones, jointly independent, and causally sufficient for the classification. Based on that, we propose a Causality Inspired Representation Learning (CIRL) algorithm that enforces the representations to satisfy the above properties and then uses them to simulate the causal factors, which yields improved generalization ability. Extensive experimental results on several widely used datasets verify the effectiveness of our approach.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑