In application domains such as healthcare, we want accurate predictive models that are also causally interpretable. In pursuit of such models, we propose a causal regularizer to steer predictive models towards causally-interpretable solutions and theoretically study its properties. In a large-scale analysis of Electronic Health Records (EHR), our causally-regularized model outperforms its L1-regularized counterpart in causal accuracy and is competitive in predictive performance. We perform non-linear causality analysis by causally regularizing a special neural network architecture. We also show that the proposed causal regularizer can be used together with neural representation learning algorithms to yield up to 20% improvement over multilayer perceptron in detecting multivariate causation, a situation common in healthcare, where many causal factors should occur simultaneously to have an effect on the target variable.
Causally-informed Deep Learning towards Explainable and Generalizable Outcomes Prediction in Critical Care
Tsinghua University（清华大学）；Institute of Brain and Cognitive Sciences, Tsinghua University（清华大学脑与认知科学研究院）；Chinese PLA General Hospital（中国人民解放军总医院）；Shanghai Artificial Intelligence Laboratory（上海人工智能实验室）
Deep neural networks excel at comprehending complex visual signals, delivering on par or even superior performance to that of human experts. However, ad-hoc visual explanations of model decisions often reveal an alarming level of reliance on exploiting non-causal visual cues that strongly correlate with the target label in training data. As such, deep neural nets suffer compromised generalization to novel inputs collected from different sources, and the reverse engineering of their decision rules offers limited interpretability. To overcome these limitations, we present a novel contrastive lea…
Graph attention networks estimate the relational importance of node neighbors to aggregate relevant information over local neighborhoods for a prediction task. However, the inferred attentions are vulnerable to spurious correlations and connectivity in the training data, hampering the generalizability of the model. We introduce CAR, a general-purpose regularization framework for graph attention networks. Embodying a causal inference approach, CAR aligns the attention mechanism with the causal effects of active interventions on graph connectivity in a scalable manner. CAR is compatible with a v…
University of California, Los Angeles（加利福尼亚大学洛杉矶分校）；University of Cambridge（剑桥大学）；University of Oxford（牛津大学）；The Alan Turing Institute（艾伦·图灵研究所）
Missing data is an important problem in machine learning practice. Starting from the premise that imputation methods should preserve the causal structure of the data, we develop a regularization scheme that encourages any baseline imputation method to be causally consistent with the underlying data generating mechanism. Our proposal is a causally-aware imputation algorithm (MIRACLE). MIRACLE iteratively refines the imputation of a baseline by simultaneously modeling the missingness generating mechanism, encouraging imputation to be consistent with the causal structure of the data. We conduct e…
Università della Svizzera Italiana（瑞士意大利大学）；University of Liechtenstein（利赫泰森大学）；Politecnico di Torino（托尼诺理工学院）；IBM Research（IBM研究院）；Scuola Universitaria Professionale della Svizzera Italiana（瑞士意大利专业大学）
Indian Institute of Technology Hyderabad（印度理工学院海得拉巴分校）
Representation learners that disentangle factors of variation have already proven to be important in addressing various real world concerns such as fairness and interpretability. Initially consisting of unsupervised models with independence assumptions, more recently, weak supervision and correlated features have been explored, but without a causal view of the generative process. In contrast, we work under the regime of a causal generative process where generative factors are either independent or can be potentially confounded by a set of observed or unobserved confounders. We present an analy…
Identifying causal relationships for a treatment intervention is a fundamental problem in health sciences. Randomized controlled trials (RCTs) are considered the gold standard for identifying causal relationships. However, recent advancements in the theory of causal inference based on the foundations of structural causal models (SCMs) have allowed the identification of causal relationships from observational data, under certain assumptions. Survival analysis provides standard measures, such as the hazard ratio, to quantify the effects of an intervention. While hazard ratios are widely used in …
A Versatile Causal Discovery Framework to Allow Causally-Related Hidden Variables
University of California San Diego（加州大学圣地亚哥分校）；Carnegie Mellon University（卡内基梅隆大学）；Mohamed bin Zayed University of Artificial Intelligence（穆罕默德·本·扎耶德人工智能大学）；KDDI Research（KDDI研究所）
University of Wisconsin-Madison（威斯康星大学麦迪逊分校）；Microsoft Research（微软研究院）
Making evidence based decisions requires data. However for real-world applications, the privacy of data is critical. Using synthetic data which reflects certain statistical properties of the original data preserves the privacy of the original data. To this end, prior works utilize differentially private data release mechanisms to provide formal privacy guarantees. However, such mechanisms have unacceptable privacy vs. utility trade-offs. We propose incorporating causal information into the training process to favorably modify the aforementioned trade-off. We theoretically prove that generative…
University of Tübingen（图宾根大学）；Technical University of Berlin（柏林工业大学）
In a sequential decision-making problem, having a structural dependency amongst the reward distributions associated with the arms makes it challenging to identify a subset of alternatives that guarantees the optimal collective outcome. Thus, besides individual actions' reward, learning the causal relations is essential to improve the decision-making strategy. To solve the two-fold learning problem described above, we develop the 'combinatorial semi-bandit framework with causally related rewards', where we model the causal relations by a directed graph in a stationary structural equation model.…
Does adding a theorem to a paper affect its chance of acceptance? Does labeling a post with the author's gender affect the post popularity? This paper develops a method to estimate such causal effects from observational text data, adjusting for confounding features of the text such as the subject or writing quality. We assume that the text suffices for causal adjustment but that, in practice, it is prohibitively high-dimensional. To address this challenge, we develop causally sufficient embeddings, low-dimensional document representations that preserve sufficient information for causal identif…
City University of Hong Kong（香港城市大学）；Friedrich Miescher Institute for Biomedical Research（弗里德里希·米歇尔生物医学研究所）
Distributional representations of words, also known as word vectors, have become crucial for modern natural language processing tasks due to their wide applications. Recently, a growing body of word vector postprocessing algorithm has emerged, aiming to render off-the-shelf word vectors even stronger. In line with these investigations, we introduce a novel word vector postprocessing scheme under a causal inference framework. Concretely, the postprocessing pipeline is realized by Half-Sibling Regression (HSR), which allows us to identify and remove confounding noise contained in word vectors. C…
CaTs and DAGs: Integrating Directed Acyclic Graphs with Transformers for Causally Constrained Predictions
Kivira Health（Kivira健康公司）；The Sense Innovation and Research Center, CHUV, Lausanne, Switzerland（The Sense创新与研究中心，CHUV，洛桑，瑞士）；Centre for Vision, Speech and Signal Processing, University of Surrey, U.K.（视觉、语音和信号处理中心，萨里大学，英国）；Institute of Psychology, University of Lausanne, Switzerland（心理学研究所，洛桑大学，瑞士）；EPFL, Switzerland（瑞士联邦理工学院）
Joint Laboratory of Data Science and Business Intelligence, School of Statistics, Southwestern University of Finance and Economics（西南财经大学统计学院数据科学与商务智能联合实验室）；University of Kansas（堪萨斯大学）；School of Computing and Artificial Intelligence, Southwest Jiaotong University（西南交通大学计算机与人工智能学院）
Causally Testing Gender Bias in LLMs: A Case Study on Occupational Bias
University of Illinois at Urbana-Champaign（伊利诺伊大学厄巴纳-香槟分校）；Columbia University（哥伦比亚大学）；RWTH Aachen（亚琛工业大学）；University of Michigan（密歇根大学）；ETH Zürich（苏黎世联邦理工学院）；Max Planck Institute（马克斯·普朗克研究所）
