Causal Mosaic: Cause-Effect Inference via Nonlinear ICA and Ensemble Method
The Graduate University for Advanced Studies（综合研究大学院大学）；The Institute of Statistical Mathematics（统计数理研究所）
Pengzhou Wu, Kenji Fukumizu
备注：
Accepted to AISTATS 2020. Camera-ready version in preparation
发表信息：
An updated version at AISTATS 2020: http://proceedings.mlr.press/v108/wu20b/wu20b.pdf. Main changes: a correction in Theorem 3 and additional explanations in Sec. 4
We address the problem of distinguishing cause from effect in bivariate setting. Based on recent developments in nonlinear independent component analysis (ICA), we train nonparametrically general nonlinear causal models that allow non-additive noise. Further, we build an ensemble framework, namely Causal Mosaic, which models a causal pair by a mixture of nonlinear models. We compare this method with other recent methods on artificial and real world benchmark datasets, and our method shows state-of-the-art performance.
University College London（伦敦大学学院）；Carnegie Mellon University（卡内基梅隆大学）；University of Helsinki（赫尔辛基大学）
We consider the problem of inferring causal relationships between two or more passively observed variables. While the problem of such causal discovery has been extensively studied especially in the bivariate setting, the majority of current methods assume a linear causal relationship, and the few methods which consider non-linear dependencies usually make the assumption of additive noise. Here, we propose a framework through which we can perform causal discovery in the presence of general non-linear relationships. The proposed method is based on recent progress in non-linear independent compon…
University of Sydney（悉尼大学）；University of Melbourne（墨尔本大学）；Carnegie Mellon University（卡内基梅隆大学）
Causal discovery witnessed significant progress over the past decades. In particular, many recent causal discovery methods make use of independent, non-Gaussian noise to achieve identifiability of the causal models. Existence of hidden direct common causes, or confounders, generally makes causal discovery more difficult; whenever they are present, the corresponding causal discovery algorithms can be seen as extensions of overcomplete independent component analysis (OICA). However, existing OICA algorithms usually make strong parametric assumptions on the distribution of independent components,…
HIIT Basic Research Unit（赫尔辛基信息技术研究所基础研究单元）；University of Helsinki（赫尔辛基大学）；Osaka University（大阪大学）
In recent years, several methods have been proposed for the discovery of causal structure from non-experimental data (Spirtes et al. 2000; Pearl 2000). Such methods make various assumptions on the data generating process to facilitate its identification from purely observational data. Continuing this line of research, we show how to discover the complete causal structure of continuous-valued data, under the assumptions that (a) the data generating process is linear, (b) there are no unobserved confounders, and (c) disturbance variables have non-gaussian distributions of non-zero variances. The…
Causal Discovery of Linear Non-Gaussian Causal Models with Unobserved Confounding
TUM School of Computation, Information and Technology（慕尼黑工业大学计算、信息与技术学院）；Technical University of Munich（慕尼黑工业大学）；University of British Columbia（不列颠哥伦比亚大学）；Munich Center for Machine Learning（慕尼黑机器学习中心）
School of Computer Science, Guangdong University of Technology（广东工业大学计算机学院）；Department of Philosophy, Carnegie Mellon University（卡内基梅隆大学哲学系）；Pazhou Lab（琶洲实验室）；School of Mathematics and Big Data, Foshan University（佛山大学数学与大数据学院）
We consider the problem of estimating a particular type of linear non-Gaussian model. Without resorting to the overcomplete Independent Component Analysis (ICA), we show that under some mild assumptions, the model is uniquely identified by a hybrid method. Our method leverages the advantages of constraint-based methods and independent noise-based methods to handle both confounded and unconfounded situations. The first step of our method uses the FCI procedure, which allows confounders and is able to produce asymptotically correct results. The results, unfortunately, usually determine very few …
California Institute of Technology（加州理工学院）；UC Irvine（加州大学欧文分校）
Discovering the complete set of causal relations among a group of variables is a challenging unsupervised learning problem. Often, this challenge is compounded by the fact that there are latent or hidden confounders. When only observational data is available, the problem is ill-posed, i.e. the causal relationships are non-identifiable unless strong modeling assumptions are made. When interventions are available, we provide guarantees on identifiability and learnability under mild assumptions. We assume a linear structural equation model (SEM) with independent latent factors and directed acycli…
Multi-View Causal Discovery without Non-Gaussianity: Identifiability and Algorithms
Inria（法国国家信息与自动化技术研究院）；CEA（法国国家科学研究中心）；University of Paris-Saclay（巴黎-萨克勒大学）；University of Carnegie Mellon（卡内基梅隆大学）；Apple（苹果公司）；University of Helsinki（赫尔辛基大学）
Estimation of Structural Causal Model via Sparsely Mixing Independent Component Analysis
The Graduate University for Advanced Studies (SOKENDAI)（综合研究大学院大学）；The Institute of Statistical Mathematics（统计数理研究所）；RIKEN Center for Advanced Intelligence Project (AIP)（日本理化学研究所先进智能研究中心）
Future University Hakodate（公立函馆未来大学）；RIKEN AIP（理化学研究所先进智能项目中心）；University College London（伦敦大学学院）；Université Paris-Saclay（巴黎-萨克雷大学）；Inria（法国国家信息与自动化研究所）；CEA（法国原子能和替代能源委员会）；Helsinki Institute for Information Technology HIIT（赫尔辛基信息技术研究所 HIIT）；University of Helsinki（赫尔辛基大学）
Nonlinear independent component analysis (ICA) is a general framework for unsupervised representation learning, and aimed at recovering the latent variables in data. Recent practical methods perform nonlinear ICA by solving a series of classification problems based on logistic regression. However, it is well-known that logistic regression is vulnerable to outliers, and thus the performance can be strongly weakened by outliers. In this paper, we first theoretically analyze nonlinear ICA models in the presence of outliers. Our analysis implies that estimation in nonlinear ICA can be seriously ha…
Mila（米拉人工智能研究所）；DIRO, Université de Montréal（蒙特利尔大学计算机科学与运筹学系）；ServiceNow Research（ServiceNow研究院）；Tübingen AI Center, University of Tübingen（蒂宾根大学蒂宾根人工智能中心）；Google Research（谷歌研究院）
This work introduces a novel principle we call disentanglement via mechanism sparsity regularization, which can be applied when the latent factors of interest depend sparsely on past latent factors and/or observed auxiliary variables. We propose a representation learning method that induces disentanglement by simultaneously learning the latent factors and the sparse causal graphical model that relates them. We develop a rigorous identifiability theory, building on recent nonlinear independent component analysis (ICA) results, that formalizes this principle and shows how the latent variables ca…
We consider linear models where $d$ potential causes $X_1,...,X_d$ are correlated with one target quantity $Y$ and propose a method to infer whether the association is causal or whether it is an artifact caused by overfitting or hidden common causes. We employ the idea that in the former case the vector of regression coefficients has 'generic' orientation relative to the covariance matrix $Σ_{XX}$ of $X$. Using an ICA based model for confounding, we show that both confounding and overfitting yield regression vectors that concentrate mainly in the space of low eigenvalues of $Σ_{XX}$.
University of Cambridge（剑桥大学）；Max Planck Institute for Intelligent Systems（马克斯·普朗克智能系统研究所）；University of Toronto（多伦多大学）；Vector Institute（向量研究所）；Alan Turing Institute（艾伦·图灵研究所）
Due to spurious correlations, machine learning systems often fail to generalize to environments whose distributions differ from the ones used at training time. Prior work addressing this, either explicitly or implicitly, attempted to find a data representation that has an invariant relationship with the target. This is done by leveraging a diverse set of training environments to reduce the effect of spurious features and build an invariant predictor. However, these methods have generalization guarantees only when both data representation and classifiers come from a linear model class. We propo…
