We propose to address the task of causal structure learning from data in a supervised manner. Existing work on learning causal directions by supervised learning is restricted to learning pairwise relation, and not well suited for whole DAG discovery. We propose a novel approach of modeling the whole DAG structure discovery as a supervised learning. To fit the problem in hand, we propose to use permutation equivariant models that align well with the problem domain. We evaluate the proposed approach extensively on synthetic graphs of size 10,20,50,100 and real data, and show promising results compared with a variety of previous approaches.
RAND Corporation（兰德公司）；Pardee RAND Graduate School（帕迪兰德研究生院）；University of Southern California（南加州大学）
Fuzzy cognitive maps (FCMs) model feedback causal relations in interwoven webs of causality and policy variables. FCMs are fuzzy signed directed graphs that allow degrees of causal influence and event occurrence. Such causal models can simulate a wide range of policy scenarios and decision processes. Their directed loops or cycles directly model causal feedback. Their nonlinear dynamics permit forward-chaining inference from input causes and policy options to output effects. Users can add detailed dynamics and feedback links directly to the causal model or infer them with statistical learning …
Stanford University（斯坦福大学）；Facebook AI Research（Facebook人工智能研究院）；University of California, Los Angeles（加州大学洛杉矶分校）
A structural equation model (SEM) is an effective framework to reason over causal relationships represented via a directed acyclic graph (DAG). Recent advances have enabled effective maximum-likelihood point estimation of DAGs from observational data. However, a point estimate may not accurately capture the uncertainty in inferring the underlying graph in practical scenarios, wherein the true DAG is non-identifiable and/or the observed dataset is limited. We propose Bayesian Causal Discovery Nets (BCD Nets), a variational inference framework for estimating a distribution over DAGs characterizi…
Causal processes in biomedicine may contain cycles, evolve over time or differ between populations. However, many graphical models cannot accommodate these conditions. We propose to model causation using a mixture of directed cyclic graphs (DAGs), where the joint distribution in a population follows a DAG at any single point in time but potentially different DAGs across time. We also introduce an algorithm called Causal Inference over Mixtures that uses longitudinal data to infer a graph summarizing the causal relations generated from a mixture of DAGs. Experiments demonstrate improved perform…
The combinatorial search space presents a significant challenge to learning causality from data. Recently, the problem has been formulated into a continuous optimization framework with an acyclicity constraint, allowing for the exploration of deep generative models to better capture data sample distributions and support the discovery of Directed Acyclic Graphs (DAGs) that faithfully represent the underlying data distribution. However, so far no study has investigated the use of Wasserstein distance for causal structure learning via generative models. This paper proposes a new model named DAG-W…
Causal structure learning with momentum: Sampling distributions over Markov Equivalence Classes of DAGs
Chalmers University of Technology（查尔姆斯理工大学）；University of Gothenburg（哥德堡大学）；Institute for Theoretical Computer Science（理论计算机科学研究所）；University of Lübeck（吕贝克大学）
Massachusetts Institute of Technology（麻省理工学院）；University of Michigan（密歇根大学）；ETH Zurich（苏黎世联邦理工学院）
We consider distributions arising from a mixture of causal models, where each model is represented by a directed acyclic graph (DAG). We provide a graphical representation of such mixture distributions and prove that this representation encodes the conditional independence relations of the mixture distribution. We then consider the problem of structure learning based on samples from such distributions. Since the mixing variable is latent, we consider causal structure discovery algorithms such as FCI that can deal with latent variables. We show that such algorithms recover a "union" of the comp…
ETH Zurich（苏黎世联邦理工学院）；Mila（米拉人工智能研究所）；Université de Montréal（蒙特利尔大学）；ElementAI；ServiceNow；MPI for Intelligent Systems（马克斯·普朗克智能系统研究所）；CIFAR（加拿大高等研究院）
Learning the causal structure that underlies data is a crucial step towards robust real-world decision making. The majority of existing work in causal inference focuses on determining a single directed acyclic graph (DAG) or a Markov equivalence class thereof. However, a crucial aspect to acting intelligently upon the knowledge about causal structure which has been inferred from finite data demands reasoning about its uncertainty. For instance, planning interventions to find out more about the causal mechanisms that govern our data requires quantifying epistemic uncertainty over DAGs. While Ba…
Università Cattolica del Sacro Cuore（天主教圣心大学）；Università degli Studi di Milano-Bicocca（米兰比可卡大学）
Directed Acyclic Graphs (DAGs) provide a powerful framework to model causal relationships among variables in multivariate settings; in addition, through the do-calculus theory, they allow for the identification and estimation of causal effects between variables also from pure observational data. In this setting, the process of inferring the DAG structure from the data is referred to as causal structure learning or causal discovery. We introduce BCDAG, an R package for Bayesian causal discovery and causal effect estimation from Gaussian observational data, implementing the Markov chain Monte Ca…
Columbia University（哥伦比亚大学）；Iowa State University（爱荷华州立大学）
Reasoning about the effect of interventions and counterfactuals is a fundamental task found throughout the data sciences. A collection of principles, algorithms, and tools has been developed for performing such tasks in the last decades (Pearl, 2000). One of the pervasive requirements found throughout this literature is the articulation of assumptions, which commonly appear in the form of causal diagrams. Despite the power of this approach, there are significant settings where the knowledge necessary to specify a causal diagram over all variables is not available, particularly in complex, high…
Causal reasoning is a crucial part of science and human intelligence. In order to discover causal relationships from data, we need structure discovery methods. We provide a review of background theory and a survey of methods for structure discovery. We primarily focus on modern, continuous optimization methods, and provide reference to further resources such as benchmark datasets and software packages. Finally, we discuss the assumptive leap required to take us from structure to causality.
University of Warwick（华威大学）；The Alan Turing Institute（艾伦·图灵研究所）；University of Sheffield（谢菲尔德大学）；Microsoft Research Cambridge（微软剑桥研究院）
This paper studies the problem of learning the correlation structure of a set of intervention functions defined on the directed acyclic graph (DAG) of a causal model. This is useful when we are interested in jointly learning the causal effects of interventions on different subsets of variables in a DAG, which is common in field such as healthcare or operations research. We propose the first multi-task causal Gaussian process (GP) model, which we call DAG-GP, that allows for information sharing across continuous interventions and across experiments on different variables. DAG-GP accommodates di…
University of Illinois at Urbana-Champaign（伊利诺伊大学厄巴纳-香槟分校）；Sharif University of Technology（谢里夫理工大学）；Georgia Institute of Technology（佐治亚理工学院）；Carnegie Mellon University（卡内基梅隆大学）
A directed acyclic graph (DAG) is the most common graphical model for representing causal relationships among a set of variables. When restricted to using only observational data, the structure of the ground truth DAG is identifiable only up to Markov equivalence, based on conditional independence relations among the variables. Therefore, the number of DAGs equivalent to the ground truth DAG is an indicator of the causal complexity of the underlying structure--roughly speaking, it shows how many interventions or how much additional information is further needed to recover the underlying DAG. I…
Estimate Collapsibility of Causal Effects in Completed Partial DAGs via Strong d-Convex Hulls
College of Mathematics and System Science, Xinjiang University（新疆大学数学与系统科学学院）；Institute of Statistics and Data Science, Xinjiang University of Finance and Economics（新疆财经大学统计与数据科学研究院）
Directed acyclic graphs (DAGs) are frequently used in epidemiology as a method to encode causal inference assumptions. We propose the DAGWOOD framework to bring many of those encoded assumptions to the forefront. DAGWOOD combines a root DAG (the DAG in the proposed analysis) and a set of branch DAGs (alternative hidden assumptions to the root DAG). All branch DAGs share a common ruleset, and must 1) change the root DAG, 2) be a valid DAG, and either 3a) change the minimally sufficient adjustment set or 3b) change the number of frontdoor paths. Branch DAGs comprise a list of assumptions which m…
Williams College（威廉姆斯学院）；Emory University（埃默里大学）；Johns Hopkins University（约翰斯·霍普金斯大学）
Identification theory for causal effects in causal models associated with hidden variable directed acyclic graphs (DAGs) is well studied. However, the corresponding algorithms are underused due to the complexity of estimating the identifying functionals they output. In this work, we bridge the gap between identification and estimation of population-level causal effects involving a single treatment and a single outcome. We derive influence function based estimators that exhibit double robustness for the identified effects in a large class of hidden variable DAGs where the treatment satisfies a …
Causal structures for observational survival data provide crucial information regarding the relationships between covariates and time-to-event. We derive motivation from the information theoretic source coding argument, and show that incorporating the knowledge of the directed acyclic graph (DAG) can be beneficial if suitable source encoders are employed. As a possible source encoder in this context, we derive a variational inference based conditional variational autoencoder for causal structured survival prediction, which we refer to as DAGSurv. We illustrate the performance of DAGSurv on low…
The problem of learning a directed acyclic graph (DAG) up to Markov equivalence is equivalent to the problem of finding a permutation of the variables that induces the sparsest graph. Without additional assumptions, this task is known to be NP-hard. Building on the minimum degree algorithm for sparse Cholesky decomposition, but utilizing DAG-specific problem structure, we introduce an efficient algorithm for finding such sparse permutations. We show that on jointly Gaussian distributions, our method with depth $w$ runs in $O(p^{w+3})$ time. We compare our method with $w = 1$ to algorithms for …
Models and Algorithms for Data and Text Mining Laboratory (MADLab) Department of Informatics Systems and Communication (DISCo) University of Milano - Bicocca Milan Italy（数据与文本挖掘算法实验室（MADLab）信息系统与通信系（DISCo）米兰-比科卡大学）；Istituto Dalle Molle di Studi sull’Intelligenza Artificiale (IDSIA) Lugano Switzerland（智能研究 institute（IDSIA）卢加诺瑞士）；University of Milano - Bicocca Milan Italy（米兰-比科卡大学）
IBM Research（IBM研究院）；MIT-IBM Watson AI Lab（麻省理工学院-IBM沃森人工智能实验室）；MIT（麻省理工学院）
Directed acyclic graph (DAG) models are popular for capturing causal relationships. From observational and interventional data, a DAG model can only be determined up to its \emph{interventional Markov equivalence class} (I-MEC). We investigate the size of MECs for random DAG models generated by uniformly sampling and ordering an Erdős-Rényi graph. For constant density, we show that the expected $\log$ observational MEC size asymptotically (in the number of vertices) approaches a constant. We characterize I-MEC size in a similar fashion in the above settings with high precision. We show that th…
LOCAL: Learning with Orientation Matrix to Infer Causal Structure from Time Series Data
School of Software Engineering, Beijing Jiaotong University（北京交通大学软件学院）；Department of Computer Science, Hong Kong Baptist University（香港浸会大学计算机科学系）；Department of Electrical and Computer Engineering, University of Alberta（阿尔伯塔大学电气与计算机工程系）
UCLA Department of Biostatistics（加州大学洛杉矶分校生物统计系）；UCLA Department of Statistics（加州大学洛杉矶分校统计系）
Discovery of causal relationships from observational data is an important problem in many areas. Several recent results have established the identifiability of causal DAGs with non-Gaussian and/or nonlinear structural equation models (SEMs). In this paper, we focus on nonlinear SEMs defined by non-invertible functions, which exist in many data domains, and propose a novel test for non-invertible bivariate causal models. We further develop a method to incorporate this test in structure learning of DAGs that contain both linear and nonlinear causal relations. By extensive numerical comparisons, …
Shandong University（山东大学）；Suzhou University of Science and Technology（苏州科技大学）；Beijing University of Civil Engineering and Architecture（北京建筑大学）
Learning directed acyclic graph (DAG) that describes the causality of observed data is a very challenging but important task. Due to the limited quantity and quality of observed data, and non-identifiability of causal graph, it is almost impossible to infer a single precise DAG. Some methods approximate the posterior distribution of DAGs to explore the DAG space via Markov chain Monte Carlo (MCMC), but the DAG space is over the nature of super-exponential growth, accurately characterizing the whole distribution over DAGs is very intractable. In this paper, we propose {Reinforcement Causal Stru…
CaTs and DAGs: Integrating Directed Acyclic Graphs with Transformers for Causally Constrained Predictions
Kivira Health（Kivira健康公司）；The Sense Innovation and Research Center, CHUV, Lausanne, Switzerland（The Sense创新与研究中心，CHUV，洛桑，瑞士）；Centre for Vision, Speech and Signal Processing, University of Surrey, U.K.（视觉、语音和信号处理中心，萨里大学，英国）；Institute of Psychology, University of Lausanne, Switzerland（心理学研究所，洛桑大学，瑞士）；EPFL, Switzerland（瑞士联邦理工学院）
Conventional methods for causal structure learning from data face significant challenges due to combinatorial search space. Recently, the problem has been formulated into a continuous optimization framework with an acyclicity constraint to learn Directed Acyclic Graphs (DAGs). Such a framework allows the utilization of deep generative models for causal structure learning to better capture the relations between data sample distributions and DAGs. However, so far no study has experimented with the use of Wasserstein distance in the context of causal structure learning. Our model named DAG-WGAN c…
Tags for DAGs: Graph Refinement with Meta-Informed Relations
Computer Science Department, Technical University of Darmstadt（达姆施塔特技术大学计算机科学系）；Hessian Center for AI (hessian.AI)（海德堡人工智能中心）；German Research Center for AI (DFKI)（德国人工智能研究中心）；Centre for Cognitive Science, Technical University of Darmstadt（达姆施塔特技术大学认知科学中心）；Department of Mathematics and Computer Science, Eindhoven University of Technology（埃因霍温理工大学数学与计算机科学系）
Personalized Binomial DAGs Learning with Network Structured Covariates
Booth School of Business, University of Chicago（芝加哥大学布斯商学院）；University of Michigan（密歇根大学）；Ant Financial Group（蚂蚁集团）；Marshall School of Business, University of Southern California（南加州大学马歇尔商学院）
A common theme in causal inference is learning causal relationships between observed variables, also known as causal discovery. This is usually a daunting task, given the large number of candidate causal graphs and the combinatorial nature of the search space. Perhaps for this reason, most research has so far focused on relatively small causal graphs, with up to hundreds of nodes. However, recent advances in fields like biology enable generating experimental data sets with thousands of interventions followed by rich profiling of thousands of variables, raising the opportunity and urgent need f…
University of Copenhagen（哥本哈根大学）；Maastricht University（马斯特里赫特大学）
Simulated DAG models may exhibit properties that, perhaps inadvertently, render their structure identifiable and unexpectedly affect structure learning algorithms. Here, we show that marginal variance tends to increase along the causal order for generically sampled additive noise models. We introduce varsortability as a measure of the agreement between the order of increasing marginal variance and the causal order. For commonly sampled graphs and model parameters, we show that the remarkable performance of some continuous structure learning algorithms can be explained by high varsortability an…
University of Cambridge（剑桥大学）；The Alan Turing Institute（艾伦·图灵研究所）；University of California Los Angeles（加州大学洛杉矶分校）
Unobserved confounding is one of the greatest challenges for causal discovery. The case in which unobserved variables have a widespread effect on many of the observed ones is particularly difficult because most pairs of variables are conditionally dependent given any other subset, rendering the causal effect unidentifiable. In this paper we show that beyond conditional independencies, under the principle of independent mechanisms, unobserved confounding in this setting leaves a statistical footprint in the observed data distribution that allows for disentangling spurious and causal effects. Us…
Department of Technology, Management and Economics, Danish Technical University（丹麦技术大学技术、管理与经济学系）；Department of Mathematics and Statistics, Binghamton University, State University of New York（纽约州立大学宾汉姆顿分校数学与统计系）；Department of Mathematical Sciences, University of Copenhagen（哥本哈根大学数学科学系）
University of Illinois at Urbana-Champaign（伊利诺伊大学厄巴纳-香槟分校）；Sharif University of Technology（谢里夫理工大学）；École Polytechnique Fédérale de Lausanne（洛桑联邦理工学院）
It is known that from purely observational data, a causal DAG is identifiable only up to its Markov equivalence class, and for many ground truth DAGs, the direction of a large portion of the edges will be remained unidentified. The golden standard for learning the causal DAG beyond Markov equivalence is to perform a sequence of interventions in the system and use the data gathered from the interventional distributions. We consider a setup in which given a budget $k$, we design $k$ interventions non-adaptively. We cast the problem of finding the best intervention target set as an optimization p…
Coarsening Linear Non-Gaussian Causal Models with Cycles
Department of Technology, Management and Economics, Technical University of Denmark（技术、管理与经济系，丹麦技术大学）；Department of Mathematical Sciences, University of Copenhagen（数学科学系，哥本哈根大学）
This paper considers inference of causal structure in a class of graphical models called "conditional DAGs". These are directed acyclic graph (DAG) models with two kinds of variables, primary and secondary. The secondary variables are used to aid in estimation of causal relationships between the primary variables. We give causal semantics for this model class and prove that, under certain assumptions, the direction of causal influence is identifiable from the joint observational distribution of the primary and secondary variables. A score-based approach is developed for estimation of causal st…
