In recent years, a lot of research has been conducted within the area of causal inference and causal learning. Many methods have been developed to identify the cause-effect pairs in models and have been successfully applied to observational real-world data to determine the direction of causal relationships. Yet in bivariate situations, causal discovery problems remain challenging. One class of such methods, that also allows tackling the bivariate case, is based on Additive Noise Models (ANMs). Unfortunately, one aspect of these methods has not received much attention until now: what is the impact of different noise levels on the ability of these methods to identify the direction of the causal relationship. This work aims to bridge this gap with the help of an empirical study. We test Regression with Subsequent Independence Test (RESIT) using an exhaustive range of models where the level of additive noise gradually changes from 1\% to 10000\% of the causes' noise level (the latter remains fixed). Additionally, the experiments in this work consider several different types of distributions as well as linear and non-linear models. The results of the experiments show that ANMs methods can fail to capture the true causal direction for some levels of noise.
Toyota Technological Institute-Chicago（芝加哥丰田理工学院）；Max Planck Institute for Intelligent Systems（马克斯·普朗克智能系统研究所）
We analyze a family of methods for statistical causal inference from sample under the so-called Additive Noise Model. While most work on the subject has concentrated on establishing the soundness of the Additive Noise Model, the statistical consistency of the resulting inference methods has received little attention. We derive general conditions under which the given family of inference methods consistently infers the causal direction in a nonparametric setting.
This paper demonstrates how to recover causal graphs from the score of the data distribution in non-linear additive (Gaussian) noise models. Using score matching algorithms as a building block, we show how to design a new generation of scalable causal discovery methods. To showcase our approach, we also propose a new efficient method for approximating the score's Jacobian, enabling to recover the causal graph. Empirically, we find that the new algorithm, called SCORE, is competitive with state-of-the-art causal discovery methods while being significantly faster.
Guangdong University of Technology（广东工业大学）；Carnegie Mellon University（卡内基梅隆大学）；Yitu Technology Ltd.（依图科技有限公司）；Foshan University（佛山科学技术学院）
Identification of causal direction between a causal-effect pair from observed data has recently attracted much attention. Various methods based on functional causal models have been proposed to solve this problem, by assuming the causal process satisfies some (structural) constraints and showing that the reverse direction violates such constraints. The nonlinear additive noise model has been demonstrated to be effective for this purpose, but the model class is not transitive--even if each direct causal relation follows this model, indirect causal influences, which result from omitted intermedi…
University of Oxford（牛津大学）；Technion – Israel Institute of Technology（以色列理工学院）
We study the problem of learning conditional average treatment effects (CATE) from high-dimensional, observational data with unobserved confounders. Unobserved confounders introduce ignorance -- a level of unidentifiability -- about an individual's response to treatment by inducing bias in CATE estimates. We present a new parametric interval estimator suited for high-dimensional data, that estimates a range of possible CATE values when given a predefined bound on the level of hidden confounding. Further, previous interval estimators do not account for ignorance about the CATE associated with s…
Neuroimaging datasets keep growing in size to address increasingly complex medical questions. However, even the largest datasets today alone are too small for training complex machine learning models. A potential solution is to increase sample size by pooling scans from several datasets. In this work, we combine 12,207 MRI scans from 15 studies and show that simple pooling is often ill-advised due to introducing various types of biases in the training data. First, we systematically define these biases. Second, we detect bias by experimentally showing that scans can be correctly assigned to the…
Causal structure discovery in complex dynamical systems is an important challenge for many scientific domains. Although data from (interventional) experiments is usually limited, large amounts of observational time series data sets are usually available. Current methods that learn causal structure from time series often assume linear relationships. Hence, they may fail in realistic settings that contain nonlinear relations between the variables. We propose Neural Additive Vector Autoregression (NAVAR) models, a neural approach to causal structure learning that can discover nonlinear relationsh…
The Chinese University of Hong Kong（香港中文大学）；Huawei Noah’s Ark Lab（华为诺亚方舟实验室）；Huawei Montréal Research Center（华为蒙特利尔研究中心）
The inference of the causal relationship between a pair of observed variables is a fundamental problem in science, and most existing approaches are based on one single causal model. In practice, however, observations are often collected from multiple sources with heterogeneous causal models due to certain uncontrollable factors, which renders causal analysis results obtained by a single model skeptical. In this paper, we generalize the Additive Noise Model (ANM) to a mixture model, which consists of a finite number of ANMs, and provide the condition of its causal identifiability. To conduct mo…
Faculty of Science, Technology and Communication（科学与技术通信学院）
In recent years a lot of research has been conducted within the area of causal inference and causal learning. Many methods have been developed to identify the cause-effect pairs in models and have been successfully applied to observational real-world data in order to determine the direction of causal relationships. Many of these methods require simplifying assumptions, such as absence of confounding, cycles, and selection bias. Yet in bivariate situations causal discovery problems remain challenging. One class of such methods, that also allows tackling the bivariate case, is based on Additive …
Estimating average causal effect (ACE) is useful whenever we want to know the effect of an intervention on a given outcome. In the absence of a randomized experiment, many methods such as stratification and inverse propensity weighting have been proposed to estimate ACE. However, it is hard to know which method is optimal for a given dataset or which hyperparameters to use for a chosen method. To this end, we provide a framework to characterize the loss of a causal inference method against the true ACE, by framing causal inference as a representation learning problem. We show that many popular…
The University of Melbourne（墨尔本大学）；Carnegie Mellon University（卡内基梅隆大学）；JD Explore Academy（京东探索研究院）；The University of Sydney（悉尼大学）；Mohamed bin Zayed University of Artificial Intelligence（穆罕默德·本·扎耶德人工智能大学）
State-of-the-art causal discovery methods usually assume that the observational data is complete. However, the missing data problem is pervasive in many practical scenarios such as clinical trials, economics, and biology. One straightforward way to address the missing data problem is first to impute the data using off-the-shelf imputation methods and then apply existing causal discovery methods. However, such a two-step method may suffer from suboptimality, as the imputation algorithm may introduce bias for modeling the underlying data distribution. In this paper, we develop a general method, …
RIKEN Center for Advanced Intelligence Project（理化学研究所先进智能项目中心）；Shiga University（滋贺大学）
Causal discovery from data affected by unobserved variables is an important but difficult problem to solve. The effects that unobserved variables have on the relationships between observed variables are more complex in nonlinear cases than in linear cases. In this study, we focus on causal additive models in the presence of unobserved variables. Causal additive models exhibit structural equations that are additive in the variables and error terms. We take into account the presence of not only unobserved common causes but also unobserved intermediate variables. Our theoretical results show that…
The notion of causality is used in many situations dealing with uncertainty. We consider the problem whether causality can be identified given data set generated by discrete random variables rather than continuous ones. In particular, for non-binary data, thus far it was only known that causality can be identified except rare cases. In this paper, we present necessary and sufficient condition for an integer modular acyclic additive noise (IMAN) of two variables. In addition, we relate bivariate and multivariate causal identifiability in a more explicit manner, and develop a practical algorithm…
A tutorial on discovering and quantifying the effect of latent causal sources of multimodal EHR data
Department of Biomedical Informatics Vanderbilt University Medical Center（生物医学信息学系范德堡大学医学中心）；Department of Biomedical Informatics University of Pittsburgh（生物医学信息学系匹兹堡大学）；Departments of Medicine & Biomedical Informatics Vanderbilt University Medical Center（医学与生物医学信息学系范德堡大学医学中心）；Departments of Biomedical Informatics & Computer Science Vanderbilt University Medical Center & Vanderbilt University（生物医学信息学与计算机科学系范德堡大学医学中心及范德堡大学）
We develop estimation for potentially high-dimensional additive structural equation models. A key component of our approach is to decouple order search among the variables from feature or edge selection in a directed acyclic graph encoding the causal structure. We show that the former can be done with nonregularized (restricted) maximum likelihood estimation while the latter can be efficiently addressed using sparse regression techniques. Thus, we substantially simplify the problem of structure search and estimation for an important class of causal models. We establish consistency of the (rest…
Istituto Italiano di Tecnologia（意大利理工学院）；Harvard Medical School（哈佛医学院）；Whiting School of Engineering（怀廷工程学院）；Johns Hopkins University（约翰斯·霍普金斯大学）
Constraint-based structure learning algorithms infer the causal structure of multivariate systems from observational data by determining an equivalent class of causal structures compatible with the conditional independencies in the data. Methods based on additive-noise (AN) models have been proposed to further discriminate between causal structures that are equivalent in terms of conditional independencies. These methods rely on a particular form of the generative functional equations, with an additive noise structure, which allows inferring the directionality of causation by testing the indep…
Spurious association arises from covariance between propensity for the treatment and individual risk for the outcome. For sensitivity analysis with stochastic counterfactuals we introduce a methodology to characterize uncertainty in causal inference from natural experiments and quasi-experiments. Our sensitivity parameters are standardized measures of variation in propensity and individual risk, and one minus their geometric mean is an intuitive measure of randomness in the data generating process. Within our latent propensity-risk model, we show how to compute from contingency table data a th…
We consider the problem of learning causal directed acyclic graphs from an observational joint distribution. One can use these graphs to predict the outcome of interventional experiments, from which data are often not available. We show that if the observational distribution follows a structural equation model with an additive noise structure, the directed acyclic graph becomes identifiable from the distribution under mild conditions. This constitutes an interesting alternative to traditional methods that assume faithfulness and identify only the Markov equivalence class of the graph, thus lea…
Many methods for causal inference generate directed acyclic graphs (DAGs) that formalize causal relations between $n$ variables. Given the joint distribution on all these variables, the DAG contains all information about how intervening on one variable changes the distribution of the other $n-1$ variables. However, quantifying the causal influence of one variable on another one remains a nontrivial question. Here we propose a set of natural, intuitive postulates that a measure of causal strength should satisfy. We then introduce a communication scenario, where edges in a DAG play the role of c…
Inferring the causal structure of a set of random variables from a finite sample of the joint distribution is an important problem in science. Recently, methods using additive noise models have been suggested to approach the case of continuous variables. In many situations, however, the variables of interest are discrete or even have only finitely many states. In this work we extend the notion of additive noise models to these cases. We prove that whenever the joint distribution $\prob^{(X,Y)}$ admits such a model in one direction, e.g. $Y=f(X)+N, N \independent X$, it does not admit the rever…
The analysis of causal effects when the outcome of interest is possibly truncated by death has a long history in statistics and causal inference. The survivor average causal effect is commonly identified with more assumptions than those guaranteed by the design of a randomized clinical trial or using sensitivity analysis. This paper demonstrates that individual level causal effects in the `always survivor' principal stratum can be identified with no stronger identification assumptions than randomization. We illustrate the practical utility of our methods using data from a clinical trial on pat…
Gatsby Unit, UCL（伦敦大学学院盖茨比单元）；King’s College London（伦敦国王学院）；University of Helsinki（赫尔辛基大学）
Two apparently unrelated fields -- normalizing flows and causality -- have recently received considerable attention in the machine learning community. In this work, we highlight an intrinsic correspondence between a simple family of autoregressive normalizing flows and identifiable causal models. We exploit the fact that autoregressive flow architectures define an ordering over variables, analogous to a causal ordering, to show that they are well-suited to performing a range of causal inference tasks, ranging from causal discovery to making interventional and counterfactual predictions. First,…
Identification of Causal Structure in the Presence of Missing Data with Additive Noise Model
Guangdong University of Technology（广东工业大学）；Peng Cheng Laboratory（鹏城实验室）；Mohamed bin Zayed University of Artificial Intelligence（穆罕默德·本·扎耶德人工智能大学）；Shantou University（汕头大学）
Quantifying Causes of Arctic Amplification via Deep Learning based Time-series Causal Inference
University of Maryland, Baltimore County（马里兰大学巴尔的摩郡分校）；University of Colorado Boulder（科罗拉多大学博尔德分校）；Geophysical Fluid Dynamics Laboratory, NOAA（美国国家海洋和大气管理局地球物理流体动力学实验室）；NSF HDR Institute for Harnessing Data and Model Revolution in the Polar Regions (iHARP)（美国国家科学基金会HDR极地数据与模型变革应用研究所（iHARP））
This paper quantifies the effect of speed cameras on road traffic collisions using an approximate Bayesian doubly-robust (DR) causal inference estimation method. Previous empirical work on this topic, which shows a diverse range of estimated effects, is based largely on outcome regression (OR) models using the Empirical Bayes approach or on simple before and after comparisons. Issues of causality and confounding have received little formal attention. A causal DR approach combines propensity score (PS) and OR models to give an average treatment effect (ATE) estimator that is consistent and asym…
A Scale-Invariant Sorting Criterion to Find a Causal Order in Additive Noise Models
CNRS, MAP5（法国国家科学研究中心，MAP5）；Université Paris Cité（巴黎西岱大学）；CentraleSupélec（中央理工-高等电力学院）；Université Paris-Saclay（巴黎-萨克雷大学）；University of Copenhagen（哥本哈根大学）
One major drawback of state-of-the-art artificial intelligence is its lack of explainability. One approach to solve the problem is taking causality into account. Causal mechanisms can be described by structural causal models. In this work, we propose a method for estimating bivariate structural causal models using a combination of normalising flows applied to density estimation and variational Gaussian process regression for post-nonlinear models. It facilitates causal discovery, i.e. distinguishing cause and effect, by either the independence of cause and residual or a likelihood ratio test. …
iSCAN: Identifying Causal Mechanism Shifts among Nonlinear Additive Noise Models
University of Chicago（芝加哥大学）；Booth School of Business, University of Chicago（芝加哥大学布斯商学院）；Carnegie Mellon University（卡内基梅隆大学）；University of Texas at Austin（德克萨斯大学奥斯汀分校）
Hazard ratios are prone to selection bias, compromising their use as causal estimands. On the other hand, the hazard difference has been shown to remain unaffected by the selection of frailty factors over time. Therefore, observed hazard differences can be used as an unbiased estimator for the causal hazard differences in the absence of confounding. However, in the presence of effect (on the hazard) heterogeneity, the hazard difference is also affected by selection. In this work, we formalize how the observed hazard difference (from a randomized controlled trial) evolves by selecting favourabl…
The discovery of non-linear causal relationship under additive non-Gaussian noise models has attracted considerable attention recently because of their high flexibility. In this paper, we propose a novel causal inference algorithm called least-squares independence regression (LSIR). LSIR learns the additive noise model through the minimization of an estimator of the squared-loss mutual information between inputs and residuals. A notable advantage of LSIR over existing approaches is that tuning parameters such as the kernel width and the regularization parameter can be naturally optimized by cr…
There is currently a dearth of appropriate methods to estimate the causal effects of multiple treatments when the outcome is binary. For such settings, we propose the use of nonparametric Bayesian modeling, Bayesian Additive Regression Trees (BART). We conduct an extensive simulation study to compare BART to several existing, propensity score-based methods and to identify its operating characteristics when estimating average treatment effects on the treated. BART consistently demonstrates low bias and mean-squared errors. We illustrate the use of BART through a comparative effectiveness analys…
Marine conservation preserves fish biodiversity, protects marine and coastal ecosystems, and supports climate resilience and adaptation. Despite the importance of establishing marine protected areas (MPAs), research on the effectiveness of MPAs with different conservation policies is limited due to the lack of quantitative MPA information. In this paper, leveraging a global MPA database, we investigate the causal impact of MPA policies on fish biodiversity. To address challenges posed by this clustered and confounded observational study, we construct a matching estimator of the average treatme…
The Graduate University for Advanced Studies（综合研究大学院大学）；The Institute of Statistical Mathematics（统计数理研究所）
We address the problem of distinguishing cause from effect in bivariate setting. Based on recent developments in nonlinear independent component analysis (ICA), we train nonparametrically general nonlinear causal models that allow non-additive noise. Further, we build an ensemble framework, namely Causal Mosaic, which models a causal pair by a mixture of nonlinear models. We compare this method with other recent methods on artificial and real world benchmark datasets, and our method shows state-of-the-art performance.
Unquantified sources of uncertainty in observational causal analyses can break the integrity of the results. One would never want another analyst to repeat a calculation with the same dataset, using a seemingly identical procedure, only to find a different conclusion. However, as we show in this work, there is a typical source of uncertainty that is essentially never considered in observational causal studies: the choice of match assignment for matched groups, that is, which unit is matched to which other unit before a hypothesis test is conducted. The choice of match assignment is anything bu…
Background: Interpreting instrumental variable results often requires further assumptions in addition to the core assumptions of relevance, independence, and the exclusion restriction. Methods: We assess whether instrument-exposure additive homogeneity renders the Wald estimand equal to the average derivative effect (ADE) in the case of a binary instrument and a continuous exposure. Results: Instrument-exposure additive homogeneity is insufficient for ADE identification when the instrument is binary, the exposure is continuous and the effect of the exposure on the outcome is non-linear on the …
Max Planck Institute for Intelligent Systems（马克斯·普朗克智能系统研究所）；University of Cambridge（剑桥大学）；Helmholtz AI（亥姆霍兹人工智能中心）；University College London（伦敦大学学院）；The Alan Turing Institute（艾伦·图灵研究所）
Causal treatment effect estimation is a key problem that arises in a variety of real-world settings, from personalized medicine to governmental policy making. There has been a flurry of recent work in machine learning on estimating causal effects when one has access to an instrument. However, to achieve identifiability, they in general require one-size-fits-all assumptions such as an additive error model for the outcome. An alternative is partial identification, which provides bounds on the causal effect. Little exists in terms of bounding methods that can deal with the most general case, wher…
Function-Valued Causal Influence in Nonlinear Time Series
Lucy Family Institute for Data \& Society, University of Notre Dame, Notre Dame, Indiana, USA.；Department of Political Science, University of Notre Dame, Notre Dame, Indiana, USA
Causal inference methods have been applied in various fields where researchers want to estimate treatment effects. In traditional causal inference settings, one assumes that the outcome of a unit does not depend on treatments of other units. However, as causal inference methods are extended to more applications, there is a greater need for estimators of general causal effects. We use an exposure mapping framework [Aronow and Samii, 2017] to map the relationship between the treatment allocation and the potential outcomes. Under the exposure model, we propose linear unbiased estimators (LUEs) fo…
Causal inference is known to be very challenging when only observational data are available. Randomized experiments are often costly and impractical and in instrumental variable regression the number of instruments has to exceed the number of causal predictors. It was recently shown in Peters et al. [2016] that causal inference for the full model is possible when data from distinct observational environments are available, exploiting that the conditional distribution of a response variable is invariant under the correct causal model. Two shortcomings of such an approach are the high computatio…
