Using deep latent variable models in causal inference has attracted considerable interest recently, but an essential open question is their ability to yield consistent causal estimates. While they have demonstrated promising results and theory exists on some simple model formulations, we also know that causal effects are not even identifiable in general with latent variables. We investigate this gap between theory and empirical results with analytical considerations and extensive experiments under multiple synthetic and real-world data sets, using the causal effect variational autoencoder (CEVAE) as a case study. While CEVAE seems to work reliably under some simple scenarios, it does not estimate the causal effect correctly with a misspecified latent variable or a complex data distribution, as opposed to its original motivation. Hence, our results show that more attention should be paid to ensuring the correctness of causal estimates with deep latent variable models.
Differentiable Causal Discovery For Latent Hierarchical Causal Models
University of California San Diego（加利福尼亚大学圣迭戈分校）；Carnegie Mellon University（卡内基梅隆大学）；Mohamed bin Zayed University of Artificial Intelligence（穆罕默德·本·扎耶德人工智能大学）
Identifiable Latent Polynomial Causal Models Through the Lens of Change
Australian Institute for Machine Learning, The University of Adelaide（阿德莱德大学澳大利亚机器学习研究所）；School of Computer Science and Engineering, The University of New South Wales（新南威尔士大学计算机科学与工程学院）；School of Mathematics and Statistics, The University of Melbourne（墨尔本大学数学与统计学院）；Mohamed bin Zayed University of Artificial Intelligence（穆罕默德·本·扎耶德人工智能大学）；Halicioğlu Data Science Institute (HDSI), University of California San Diego（加州大学圣地亚哥分校哈利西奥格鲁数据科学研究所）；Carnegie Mellon University（卡内基梅隆大学）
Graduate School of Information Science and Technology, The University of Tokyo（东京大学信息科学与技术研究生院）；Graduate School of Arts and Sciences, The University of Tokyo（东京大学文理研究生院）
California Institute of Technology（加州理工学院）；UC Irvine（加州大学欧文分校）
Discovering the complete set of causal relations among a group of variables is a challenging unsupervised learning problem. Often, this challenge is compounded by the fact that there are latent or hidden confounders. When only observational data is available, the problem is ill-posed, i.e. the causal relationships are non-identifiable unless strong modeling assumptions are made. When interventions are available, we provide guarantees on identifiability and learnability under mild assumptions. We assume a linear structural equation model (SEM) with independent latent factors and directed acycli…
Causal discovery from data affected by latent confounders is an important and difficult challenge. Causal functional model-based approaches have not been used to present variables whose relationships are affected by latent confounders, while some constraint-based methods can present them. This paper proposes a causal functional model-based method called repetitive causal discovery (RCD) to discover the causal structure of observed variables affected by latent confounders. RCD repeats inferring the causal directions between a small number of observed variables and determines whether the relatio…
Responsible AI Research Centre（负责任人工智能研究中心）；Australian Institute for Machine Learning（澳大利亚机器学习研究所）；School of Computer Science and Engineering（计算机科学与工程学院）；School of Mathematics and Statistics（数学与统计学学院）；Halicioğlu Data Science Institute（Halicioğlu数据科学研究所）；Department of Philosophy（哲学系）
Spotify（声田）；University College London（伦敦大学学院）；Babylon Health（巴比伦健康）
The discovery of causal relationships is a fundamental problem in science and medicine. In recent years, many elegant approaches to discovering causal relationships between two variables from observational data have been proposed. However, most of these deal only with purely directed causal relationships and cannot detect latent common causes. Here, we devise a general heuristic which takes a causal discovery algorithm that can only distinguish purely directed causal relations and modifies it to also detect latent common causes. We apply our method to two directed causal discovery algorithms, …
Guangdong University of Technology（广东工业大学）；Peking University（北京大学）；Pazhou Lab（琶洲实验室）；Carnegie Mellon University（卡内基梅隆大学）；Foshan University（佛山大学）
Causal discovery aims to recover causal structures or models underlying the observed data. Despite its success in certain domains, most existing methods focus on causal relations between observed variables, while in many scenarios the observed ones may not be the underlying causal variables (e.g., image pixels), but are generated by latent causal variables or confounders that are causally related. To this end, in this paper, we consider Linear, Non-Gaussian Latent variable Models (LiNGLaMs), in which latent confounders are also causally related, and propose a Generalized Independent Noise (GIN…
Guangdong University of Technology（广东工业大学）；RIKEN（理化学研究所）；Shiga University（滋贺大学）；Pazhou Lab（琶洲实验室）；Peking University（北京大学）；Okayama University（冈山大学）；Foshan University（佛山科学技术学院）
Discovering causal structures among latent factors from observed data is a particularly challenging problem. Despite some efforts for this problem, existing methods focus on the single-domain data only. In this paper, we propose Multi-Domain Linear Non-Gaussian Acyclic Models for Latent Factors (MD-LiNA), where the causal structure among latent factors of interest is shared for all domains, and we provide its identification results. The model enriches the causal representation for multi-domain data. We propose an integrated two-phase algorithm to estimate the model. In particular, we first loc…
Score-Based Causal Discovery of Latent Variable Causal Models
Carnegie Mellon University（卡内基梅隆大学）；University of California, San Diego（加州大学圣地亚哥分校）；Mohamed bin Zayed University of Artificial Intelligence（穆罕默德·本·扎耶德人工智能大学）
Carnegie Mellon University（卡内基梅隆大学）；Beijing Technology and Business University（北京工商大学）；Mohamed bin Zayed University of Artificial Intelligence（穆罕默德·本·扎耶德人工智能大学）
Most causal discovery procedures assume that there are no latent confounders in the system, which is often violated in real-world problems. In this paper, we consider a challenging scenario for causal structure identification, where some variables are latent and they form a hierarchical graph structure to generate the measured variables; the children of latent variables may still be latent and only leaf nodes are measured, and moreover, there can be multiple paths between every pair of variables (i.e., it is beyond tree structure). We propose an estimation procedure that can efficiently locate…
Temporal Latent Variable Structural Causal Model for Causal Discovery under External Interferences
School of Computer Science, Guangdong University of Technology（广东工业大学计算机科学学院）；Peng Cheng Laboratory（鹏城实验室）；Machine Learning Department, Mohamed bin Zayed University of Artificial Intelligence（Mohamed bin Zayed人工智能大学机器学习系）；College of Mathematics and Computer Science, Shantou University（汕头大学数学与计算机科学学院）
Sharif University of Technology（谢里夫理工大学）；University of Illinois at Urbana-Champaign（伊利诺伊大学厄巴纳-香槟分校）；Georgia Institute of Technology（佐治亚理工学院）；Carnegie Mellon University（卡内基梅隆大学）
We consider the problem of learning causal models from observational data generated by linear non-Gaussian acyclic causal models with latent variables. Without considering the effect of latent variables, one usually infers wrong causal relationships among the observed variables. Under faithfulness assumption, we propose a method to check whether there exists a causal path between any two observed variables. From this information, we can obtain the causal order among them. The next question is then whether or not the causal effects can be uniquely identified as well. It can be shown that causal…
Causal processes in nature may contain cycles, and real datasets may violate causal sufficiency as well as contain selection bias. No constraint-based causal discovery algorithm can currently handle cycles, latent variables and selection bias (CLS) simultaneously. I therefore introduce an algorithm called Cyclic Causal Inference (CCI) that makes sound inferences with a conditional independence oracle under CLS, provided that we can represent the cyclic causal process as a non-recursive linear structural equation model with independent errors. Empirical results show that CCI outperforms CCD in …
Our goal is to recover time-delayed latent causal variables and identify their relations from measured temporal data. Estimating causally-related latent variables from observations is particularly challenging as the latent variables are not uniquely recoverable in the most general case. In this work, we consider both a nonparametric, nonstationary setting and a parametric setting for the latent processes and propose two provable conditions under which temporally causal latent processes can be identified from their nonlinear mixtures. We propose LEAP, a theoretically-grounded framework that ext…
Most causal discovery algorithms find causal structure among a set of observed variables. Learning the causal structure among latent variables remains an important open problem, particularly when using high-dimensional data. In this paper, we address a problem for which it is known that inputs cause outputs, and these causal relationships are encoded by a causal network among a set of an unknown number of latent variables. We developed a deep learning model, which we call a redundant input neural network (RINN), with a modified architecture and a regularized objective function to find causal r…
Mila - Québec AI Institute（米拉-魁北克人工智能研究所）；McGill University（麦吉尔大学）；ÉTS Montréal（蒙特利尔高等技术学院）
Learning predictors that do not rely on spurious correlations involves building causal representations. However, learning such a representation is very challenging. We, therefore, formulate the problem of learning a causal representation from high dimensional data and study causal recovery with synthetic data. This work introduces a latent variable decoder model, Decoder BCD, for Bayesian causal discovery and performs experiments in mildly supervised and unsupervised settings. We present a series of synthetic experiments to characterize important factors for causal discovery and show that usin…
Max Planck Institute for Biological Cybernetics（马克斯·普朗克生物控制论研究所）
In nonlinear latent variable models or dynamic models, if we consider the latent variables as confounders (common causes), the noise dependencies imply further relations between the observed variables. Such models are then closely related to causal discovery in the presence of nonlinear confounders, which is a challenging problem. However, generally in such models the observation noise is assumed to be independent across data dimensions, and consequently the noise dependencies are ignored. In this paper we focus on the Gaussian process latent variable model (GPLVM), from which we develop an ex…
University of Vienna（维也纳大学）；Vienna Cognitive Science Hub（维也纳认知科学中心）
We consider the task of causal structure learning over measurement dependence inducing latent (MeDIL) causal models. We show that this task can be framed in terms of the graph theoretic problem of finding edge clique covers,resulting in an algorithm for returning minimal MeDIL causal models (minMCMs). This algorithm is non-parametric, requiring no assumptions about linearity or Gaussianity. Furthermore, despite rather weak assumptions aboutthe class of MeDIL causal models, we show that minimality in minMCMs implies some rather specific and interesting properties. By establishing MeDIL causal m…
A tutorial on discovering and quantifying the effect of latent causal sources of multimodal EHR data
Department of Biomedical Informatics Vanderbilt University Medical Center（生物医学信息学系范德堡大学医学中心）；Department of Biomedical Informatics University of Pittsburgh（生物医学信息学系匹兹堡大学）；Departments of Medicine & Biomedical Informatics Vanderbilt University Medical Center（医学与生物医学信息学系范德堡大学医学中心）；Departments of Biomedical Informatics & Computer Science Vanderbilt University Medical Center & Vanderbilt University（生物医学信息学与计算机科学系范德堡大学医学中心及范德堡大学）
Informatics Institute（信息学研究所）；University of Amsterdam（阿姆斯特丹大学）
We address the problem of causal discovery from data, making use of the recently proposed causal modeling framework of modular structural causal models (mSCM) to handle cycles, latent confounders and non-linearities. We introduce σ-connection graphs (σ-CG), a new class of mixed graphs (containing undirected, bidirected and directed edges) with additional structure, and extend the concept of σ-separation, the appropriate generalization of the well-known notion of d-separation in this setting, to apply to σ-CGs. We prove the closedness of σ-separation under marginalisation and conditioning and e…
Mila（米拉研究所）；McGill University（麦吉尔大学）；KTH Stockholm（斯德哥尔摩皇家理工学院）；ÉTS Montréal（蒙特利尔高等工程技术学院）；Deepmind（深度思维）；Université de Montréal（蒙特利尔大学）；ÉTS Montreal（蒙特利尔高等工程技术学院）
Causal learning has long concerned itself with the accurate recovery of underlying causal mechanisms. Such causal modelling enables better explanations of out-of-distribution data. Prior works on causal learning assume that the high-level causal variables are given. However, in machine learning tasks, one often operates on low-level data like image pixels or high-dimensional vectors. In such settings, the entire Structural Causal Model (SCM) -- structure, parameters, \textit{and} high-level causal variables -- is unobserved and needs to be learnt from low-level data. We treat this problem as B…
Universitas Islam Indonesia（印度尼西亚伊斯兰大学）；Radboud University Nijmegen（奈梅亨拉德堡德大学）
In our previous study, we introduced stable specification search for cross-sectional data (S3C). It is an exploratory causal method that combines stability selection concept and multi-objective optimization to search for stable and parsimonious causal structures across the entire range of model complexities. In this study, we extended S3C to S3C-Latent, to model causal relations between latent variables. We evaluated S3C-Latent on simulated data and compared the results to those of PC-MIMBuild, an extension of the PC algorithm, the state-of-the-art causal discovery method. The comparison showe…
University of Chicago（芝加哥大学）；Carnegie Mellon University（卡内基梅隆大学）
We study the problem of reconstructing a causal graphical model from data in the presence of latent variables. The main problem of interest is recovering the causal structure over the latent variables while allowing for general, potentially nonlinear dependence between the variables. In many practical problems, the dependence between raw observations (e.g. pixels in an image) is much less relevant than the dependence between certain high-level, latent features (e.g. concepts or objects), and this is the setting of interest. We provide conditions under which both the latent representations and …
University of Amsterdam（阿姆斯特丹大学）；TNO Intelligent Imaging（荷兰应用科学研究组织智能成像）；New York University（纽约大学）；CIMS（库朗数学科学研究所）；Massachusetts Institute of Technology（麻省理工学院）；CSAIL & IMES（计算机科学与人工智能实验室与医学工程与科学研究所）；University of Toronto（多伦多大学）；CIFAR（加拿大高等研究院）
Learning individual-level causal effects from observational data, such as inferring the most effective medication for a specific patient, is a problem of growing importance for policy makers. The most important aspect of inferring causal effects from observational data is the handling of confounders, factors that affect both an intervention and its outcome. A carefully designed observational study attempts to measure all important confounders. However, even if one does not have direct access to all confounders, there may exist noisy and uncertain measurement of proxies for confounders. We buil…
Causal Effect Estimation using identifiable Variational AutoEncoder with Latent Confounders and Post-Treatment Variables
College of Informatics, Huazhong Agricultural University（华中农业大学信息学院）；Data 61, Commonwealth Scientific and Industrial Research Organisation (CSIRO)（联邦科学与工业研究组织数据61分部）；STEM, University of South Australia（南澳大学 STEM 学院）
Identification of Causal Structure with Latent Variables Based on Higher Order Cumulants
Guangdong University of Technology（广东工业大学）；Peng Cheng Laboratory（鹏城实验室）；Shantou University（汕头大学）；Carnegie Mellon University（卡内基梅隆大学）；Mohamed bin Zayed University of Artificial Intelligence（穆罕默德·本·扎耶德人工智能大学）
LUMLIS (Lisbon ELLIS Unit)（里斯本ELLIS单元LUMLIS）；Instituto de Telecomunicações（电信研究所）；Unbabel（Unbabel公司）
Recent work has shown promising results in causal discovery by leveraging interventional data with gradient-based methods, even when the intervened variables are unknown. However, previous work assumes that the correspondence between samples and interventions is known, which is often unrealistic. We envision a scenario with an extensive dataset sampled from multiple intervention distributions and one observation distribution, but where we do not know which distribution originated each sample and how the intervention affected the system, \textit{i.e.}, interventions are entirely latent. We prop…
Microsoft Research（微软研究院）；University of Cambridge（剑桥大学）
Latent confounding has been a long-standing obstacle for causal reasoning from observational data. One popular approach is to model the data using acyclic directed mixed graphs (ADMGs), which describe ancestral relations between variables using directed and bidirected edges. However, existing methods using ADMGs are based on either linear functional assumptions or a discrete search that is complicated to use and lacks computational tractability for large datasets. In this work, we further extend the existing body of work and develop a novel gradient-based approach to learning an ADMG with non-…
Causal inference methods based on conditional independence construct Markov equivalent graphs, and cannot be applied to bivariate cases. The approaches based on independence of cause and mechanism state, on the contrary, that causal discovery can be inferred for two observations. In our contribution, we challenge to reconcile these two research directions. We study the role of latent variables such as latent instrumental variables and hidden common causes in the causal graphical structures. We show that the methods based on the independence of cause and mechanism, indirectly contain traces of …
Identification of Nonlinear Latent Hierarchical Models
Carnegie Mellon University（卡内基梅隆大学）；University of California San Diego（加州大学圣迭戈分校）；Beijing Technology and Business University（北京工商大学）；Mohamed bin Zayed University of Artificial Intelligence（穆罕默德·本·扎耶德人工智能大学）
Causal Discovery with Latent Confounders Based on Higher-Order Cumulants
Guangdong University of Technology（广东工业大学）；Peng Cheng Laboratory（鹏城实验室）；Shantou University（汕头大学）；Carnegie Mellon University（卡内基梅隆大学）；Mohamed bin Zayed University of Artificial Intelligence（穆罕默德·本·扎耶德人工智能大学）
Informatics Institute（信息学研究所）；University of Amsterdam（阿姆斯特丹大学）
We prove the main rules of causal calculus (also called do-calculus) for i/o structural causal models (ioSCMs), a generalization of a recently proposed general class of non-/linear structural causal models that allow for cycles, latent confounders and arbitrary probability distributions. We also generalize adjustment criteria and formulas from the acyclic setting to the general one (i.e. ioSCMs). Such criteria then allow to estimate (conditional) causal effects from observational data that was (partially) gathered under selection bias and cycles. This generalizes the backdoor criterion, the se…
A Recursive Decomposition Framework for Causal Structure Learning in the Presence of Latent Variables
Shenzhen Institute of Advanced Technology, Chinese Academy of Sciences, Shenzhen, China（深圳先进技术研究院，中国科学院，中国深圳）；College of Computer Science and Artificial Intelligence, Fudan University, Shanghai, China（复旦大学计算机科学与人工智能学院，中国上海）
Reconstructing the causal relationships behind the phenomena we observe is a fundamental challenge in all areas of science. Discovering causal relationships through experiments is often infeasible, unethical, or expensive in complex systems. However, increases in computational power allow us to process the ever-growing amount of data that modern science generates, leading to an emerging interest in the causal discovery problem from observational data. This work evaluates the LPCMCI algorithm, which aims to find generators compatible with a multi-dimensional, highly autocorrelated time series w…
University of Oxford, United Kingdom（牛津大学）；Queen Mary University of London, United Kingdom（伦敦玛丽女王大学）；Peking University, China（北京大学）；University of Bristol, United Kingdom（布里斯托大学）；King's College London, United Kingdom（伦敦国王学院）；Hong Kong University of Science（香港科学大学）；University College London, United Kingdom（伦敦大学学院）
Granger causality analysis, as one of the most popular time series causality methods, has been widely used in the economics, neuroscience. However, unobserved confounders is a fundamental problem in the observational studies, which is still not solved for the non-linear Granger causality. The application works often deal with this problem in virtue of the proxy variables, who can be treated as a measure of the confounder with noise. But the proxy variables has been proved to be unreliable, because of the bias it may induce. In this paper, we try to "recover" the unobserved confounders for the …
Harvard School of Public Health（哈佛公共卫生学院）；University of Washington（华盛顿大学）
Probabilistic inference in graphical models is the task of computing marginal and conditional densities of interest from a factorized representation of a joint probability distribution. Inference algorithms such as variable elimination and belief propagation take advantage of constraints embedded in this factorization to compute such densities efficiently. In this paper, we propose an algorithm which computes interventional distributions in latent variable causal models represented by acyclic directed mixed graphs(ADMGs). To compute these distributions efficiently, we take advantage of a recur…
University of Toronto（多伦多大学）；Vector Institute（向量研究所）
How do we learn from biased data? Historical datasets often reflect historical prejudices; sensitive or protected attributes may affect the observed treatments and outcomes. Classification algorithms tasked with predicting outcomes accurately from these datasets tend to replicate these biases. We advocate a causal modeling approach to learning from biased data, exploring the relationship between fair classification and intervention. We propose a causal model in which the sensitive attribute confounds both the treatment and the outcome. Building on prior work in deep learning and generative mod…
Korea Advanced Institute of Science and Technology (KAIST)（韩国科学技术院）
The problem of fair classification can be mollified if we develop a method to remove the embedded sensitive information from the classification features. This line of separating the sensitive information is developed through the causal inference, and the causal inference enables the counterfactual generations to contrast the what-if case of the opposite sensitive attribute. Along with this separation with the causality, a frequent assumption in the deep latent causal model defines a single latent variable to absorb the entire exogenous uncertainty of the causal graph. However, we claim that su…
University of Massachusetts, Amherst（马萨诸塞大学阿默斯特分校）；Massachusetts Institute of Technology（麻省理工学院）
Latent confounders---unobserved variables that influence both treatment and outcome---can bias estimates of causal effects. In some cases, these confounders are shared across observations, e.g. all students taking a course are influenced by the course's difficulty in addition to any educational interventions they receive individually. This paper shows how to semiparametrically model latent confounders that have this structure and thereby improve estimates of causal effects. The key innovations are a hierarchical Bayesian model, Gaussian processes with structured latent confounders (GP-SLC), an…
A Versatile Causal Discovery Framework to Allow Causally-Related Hidden Variables
University of California San Diego（加州大学圣地亚哥分校）；Carnegie Mellon University（卡内基梅隆大学）；Mohamed bin Zayed University of Artificial Intelligence（穆罕默德·本·扎耶德人工智能大学）；KDDI Research（KDDI研究所）
Centre d’Analyse et de Mathématique Sociales（社会分析与数学中心）；EHESS（高等社会科学研究院）；Centre de Mathématiques Appliquées（应用数学中心）；École Polytechnique（巴黎综合理工学院）；Inria XPOP（法国国家信息与自动化研究所XPOP团队）；Google Research（谷歌研究院）
Inferring causal effects of a treatment, intervention or policy from observational data is central to many applications. However, state-of-the-art methods for causal inference seldom consider the possibility that covariates have missing values, which is ubiquitous in many real-world analyses. Missing data greatly complicate causal inference procedures as they require an adapted unconfoundedness hypothesis which can be difficult to justify in practice. We circumvent this issue by considering latent confounders whose distribution is learned through variational autoencoders adapted to missing val…
