Rethinking Causal Relationships Learning in Graph Neural Networks
Science and Technology on Integrated Information System Laboratory, Institute of Software Chinese Academy of Sciences（中国科学院软件研究所综合信息系统技术国家级重点实验室）；University of Chinese Academy of Sciences（中国科学院大学）；State Key Laboratory of Intelligent Game（智能博弈国家重点实验室）；Department of Computer Science and Technology, Tsinghua University（清华大学计算机科学与技术系）
Hang Gao, Chengyu Yao, Jiangmeng Li, Lingyu Si, Yifan Jin, Fengge Wu, Changwen Zheng, Huaping Liu
Graph Neural Networks (GNNs) demonstrate their significance by effectively modeling complex interrelationships within graph-structured data. To enhance the credibility and robustness of GNNs, it becomes exceptionally crucial to bolster their ability to capture causal relationships. However, despite recent advancements that have indeed strengthened GNNs from a causal learning perspective, conducting an in-depth analysis specifically targeting the causal modeling prowess of GNNs remains an unresolved issue. In order to comprehensively analyze various GNN models from a causal learning perspective, we constructed an artificially synthesized dataset with known and controllable causal relationships between data and labels. The rationality of the generated data is further ensured through theoretical foundations. Drawing insights from analyses conducted using our dataset, we introduce a lightweight and highly adaptable GNN module designed to strengthen GNNs' causal learning capabilities across a diverse range of tasks. Through a series of experiments conducted on both synthetic datasets and other real-world datasets, we empirically validate the effectiveness of the proposed module.
Rethinking Dimensional Rationale in Graph Contrastive Learning from Causal Perspective
Science & Technology on Integrated Information System Laboratory, Institute of Software Chinese Academy of Sciences（中国科学院软件研究所综合信息系统技术国家级重点实验室）；State Key Laboratory of Intelligent Game（智能博弈国家重点实验室）；University of Chinese Academy of Sciences（中国科学院大学）；State Key Laboratory of Computer Science, Institute of Software Chinese Academy of Sciences（中国科学院软件研究所计算机科学国家重点实验室）
Latent variables may lead to spurious relationships that can be misinterpreted as causal relationships. In Bayesian Networks (BNs), this challenge is known as learning under causal insufficiency. Structure learning algorithms that assume causal insufficiency tend to reconstruct the ancestral graph of a BN, where bi-directed edges represent confounding and directed edges represent direct or ancestral relationships. This paper describes a hybrid structure learning algorithm, called CCHM, which combines the constraint-based part of cFCI with hill-climbing score-based learning. The score-based pro…
Hefei University of Technology（合肥工业大学）；University of South Australia（南澳大学）；Mininglamp Academy of Sciences, Mininglamp Technology（明略科技明略科学院）
Feature selection is a crucial preprocessing step in data analytics and machine learning. Classical feature selection algorithms select features based on the correlations between predictive features and the class variable and do not attempt to capture causal relationships between them. It has been shown that the knowledge about the causal relationships between features and the class variable has potential benefits for building interpretable and robust prediction models, since causal relationships imply the underlying mechanism of a system. Consequently, causality-based feature selection has gr…
German Center for Neurodegenerative Diseases (DZNE)（德国神经退行性疾病中心（DZNE））
We consider learning ancestral causal relationships in high dimensions. Our approach is driven by a supervised learning perspective, with discrete indicators of causal relationships treated as labels to be learned from available data. We focus on the setting in which some causal (ancestral) relationships are known (via background knowledge or experimental data) and put forward a general approach that scales to large problems. This is motivated by problems in human biology which are characterized by high dimensionality and potentially many latent variables. We present a case study involving int…
Indian Institute of Technology Hyderabad（印度理工学院海得拉巴分校）；Microsoft Research（微软研究院）
A trained neural network can be interpreted as a structural causal model (SCM) that provides the effect of changing input variables on the model's output. However, if training data contains both causal and correlational relationships, a model that optimizes prediction accuracy may not necessarily learn the true causal relationships between input and output variables. On the other hand, expert users often have prior knowledge of the causal relationship between certain input variables and output from domain knowledge. Therefore, we propose a regularization method that aligns the learned causal e…
Rethinking State Disentanglement in Causal Reinforcement Learning
Australia Institute for Machine Learning, The University of Adelaide（阿德莱德大学澳大利亚机器学习研究所）；Shanghai Jiao Tong University（上海交通大学）；University of California San Diego（加利福尼亚大学圣迭戈分校）；The University of Melbourne（墨尔本大学）
Discovering causal relationships between different variables from time series data has been a long-standing challenge for many domains such as climate science, finance, and healthcare. Given the complexity of real-world relationships and the nature of observations in discrete time, causal discovery methods need to consider non-linear relations between variables, instantaneous effects and history-dependent noise (the change of noise distribution due to past actions). However, previous works do not offer a solution addressing all these problems together. In this paper, we propose a novel causal …
Due to unmeasured confounding, it is often not possible to identify causal effects from a postulated model. Nevertheless, we can ask for partial identification, which usually boils down to finding upper and lower bounds of a causal quantity of interest derived from all solutions compatible with the encoded structural assumptions. One appealing way to derive such bounds is by casting it in terms of a constrained optimization method that searches over all causal models compatible with evidence, as introduced in the classic work of Balke and Pearl (1994) for discrete data. Although by constructio…
University College London（伦敦大学学院）；Carnegie Mellon University（卡内基梅隆大学）；University of Helsinki（赫尔辛基大学）
We consider the problem of inferring causal relationships between two or more passively observed variables. While the problem of such causal discovery has been extensively studied especially in the bivariate setting, the majority of current methods assume a linear causal relationship, and the few methods which consider non-linear dependencies usually make the assumption of additive noise. Here, we propose a framework through which we can perform causal discovery in the presence of general non-linear relationships. The proposed method is based on recent progress in non-linear independent compon…
How do people actively learn to learn? That is, how and when do people choose actions that facilitate long-term learning and choosing future actions that are more informative? We explore these questions in the domain of active causal learning. We propose a hierarchical Bayesian model that goes beyond past models by predicting that people pursue information not only about the causal relationship at hand but also about causal overhypotheses$\unicode{x2014}$abstract beliefs about causal relationships that span multiple situations and constrain how we learn the specifics in each situation. In two …
UCLA Department of Biostatistics（加州大学洛杉矶分校生物统计系）；UCLA Department of Statistics（加州大学洛杉矶分校统计系）
Discovery of causal relationships from observational data is an important problem in many areas. Several recent results have established the identifiability of causal DAGs with non-Gaussian and/or nonlinear structural equation models (SEMs). In this paper, we focus on nonlinear SEMs defined by non-invertible functions, which exist in many data domains, and propose a novel test for non-invertible bivariate causal models. We further develop a method to incorporate this test in structure learning of DAGs that contain both linear and nonlinear causal relations. By extensive numerical comparisons, …
KTH Royal Institute of Technology（瑞典皇家理工学院）；Carnegie Mellon University（卡内基梅隆大学）；Karolinska Institute（卡罗林斯卡学院）
Applying machine learning in the health care domain has shown promising results in recent years. Interpretable outputs from learning algorithms are desirable for decision making by health care personnel. In this work, we explore the possibility of utilizing causal relationships to refine diagnostic prediction. We focus on the task of diagnostic prediction using discomfort drawings, and explore two ways to employ causal identification to improve the diagnostic results. Firstly, we use causal identification to infer the causal relationships among diagnostic labels which, by itself, provides inte…
Digital University Kerala (IIITMK)（喀拉拉数字大学（印度信息技术与管理学院喀拉拉邦分院））
In recent years, causal modelling has been used widely to improve generalization and to provide interpretability in machine learning models. To determine cause-effect relationships in the absence of a randomized trial, we can model causal systems with counterfactuals and interventions given enough domain knowledge. However, there are several cases where domain knowledge is almost absent and the only recourse is using a statistical method to estimate causal relationships. While there have been several works done in estimating causal relationships in unstructured data, we are yet to find a well-…
A novel approach to the relationships between data features -- based on comprehensive examination of mathematical, technological, and causal methodology
Legal Research Institute of Korea University（高丽大学法律研究所）；Human-Inspired AI Research, Korea University（高丽大学类人人工智能研究院）
Identifying the causal structure of systems with multiple dynamic elements is critical to several scientific disciplines. The conventional approach is to conduct statistical tests of causality, for example with Granger Causality, between observed signals that are selected a priori. Here it is posited that, in many systems, the causal relations are embedded in a latent space that is expressed in the observed data as a linear mixture. A technique for blindly identifying the latent sources is presented: the observations are projected into pairs of components -- driving and driven -- to maximize t…
Causal Graph Learning via Distributional Invariance of Cause-Effect Relationship
The University of Tokyo, Japan（东京大学）；Institute for AI Innovation and Societal Impact (AI4LIFE)（人工智能创新与社会影响研究所）；Hanoi University of Science and Technology（河内科学技术大学）；National Institute of Advanced Industrial Science and Technology (AIST)（国家先进工业科学与技术研究院）；School of Electrical Engineering and Computer Science, Voiland College of Engineering and Architecture, Washington State University, Pullman, Washington, US（电气工程与计算机科学学院，工程与建筑学院，华盛顿州立大学）；RIKEN（日本理化学研究所）
A significant challenge in reinforcement learning is quantifying the complex relationship between actions and long-term rewards. The effects may manifest themselves over a long sequence of state-action pairs, making them hard to pinpoint. In this paper, we propose a method to link transitions with significant deviations in state with unusually large variations in subsequent rewards. Such transitions are marked as possible causal effects, and the corresponding state-action pairs are added to a separate replay buffer. In addition, we include \textit{contrastive} samples corresponding to transiti…
Estimation of causal effects is fundamental in situations were the underlying system will be subject to active interventions. Part of building a causal inference engine is defining how variables relate to each other, that is, defining the functional relationship between variables given conditional dependencies. In this paper, we deviate from the common assumption of linear relationships in causal models by making use of neural autoregressive density estimators and use them to estimate causal effects within the Pearl's do-calculus framework. Using synthetic data, we show that the approach can r…
Rethinking the Bias of Foundation Model under Long-tailed Distribution
Gaoling School of Artificial Intelligence, Renmin University of China（中国人民大学耿丽人工智能学院）；Beijing Key Laboratory of Big Data Management and Analysis Methods（北京大数据管理与分析方法重点实验室）；Institute of Software Chinese Academy of Sciences（中国科学院软件研究所）；Electrical and Computer Engineering, Carnegie Mellon University（卡内基梅隆大学电气与计算机工程学院）；University of Chinese Academy of Sciences（中国科学院大学）
Spotify（声田）；University College London（伦敦大学学院）；Babylon Health（巴比伦健康）
The discovery of causal relationships is a fundamental problem in science and medicine. In recent years, many elegant approaches to discovering causal relationships between two variables from observational data have been proposed. However, most of these deal only with purely directed causal relationships and cannot detect latent common causes. Here, we devise a general heuristic which takes a causal discovery algorithm that can only distinguish purely directed causal relations and modifies it to also detect latent common causes. We apply our method to two directed causal discovery algorithms, …
Indian Institute of Technology Kharagpur（印度理工学院卡拉格普尔分校）
We aim to mine temporal causal sequences that explain observed events (consequents) in time-series traces. Causal explanations of key events in a time-series has applications in design debugging, anomaly detection, planning, root-cause analysis and many more. We make use of decision trees and interval arithmetic to mine sequences that explain defining events in the time-series. We propose modified decision tree construction metrics to handle the non-determinism introduced by the temporal dimension. The mined sequences are expressed in a readable temporal logic language that is easy to interpre…
Identifying causal relationships for a treatment intervention is a fundamental problem in health sciences. Randomized controlled trials (RCTs) are considered the gold standard for identifying causal relationships. However, recent advancements in the theory of causal inference based on the foundations of structural causal models (SCMs) have allowed the identification of causal relationships from observational data, under certain assumptions. Survival analysis provides standard measures, such as the hazard ratio, to quantify the effects of an intervention. While hazard ratios are widely used in …
On the Relationship Between Explanation and Prediction: A Causal View
MPI for Intelligent Systems（马克斯·普朗克智能系统研究所）；ETH Zurich（苏黎世联邦理工学院）；Google Research（谷歌研究院）；CISPA-Helmholtz Center for Information Security（CISPA亥姆霍兹信息安全中心）
As machine learning models become increasingly prevalent in motion forecasting for autonomous vehicles (AVs), it is critical to ensure that model predictions are safe and reliable. However, exhaustively collecting and labeling the data necessary to fully test the long tail of rare and challenging scenarios is difficult and expensive. In this work, we construct a new benchmark for evaluating and improving model robustness by applying perturbations to existing data. Specifically, we conduct an extensive labeling effort to identify causal agents, or agents whose presence influences human drivers'…
Gene regulatory networks play a crucial role in controlling an organism's biological processes, which is why there is significant interest in developing computational methods that are able to extract their structure from high-throughput genetic data. Many of these computational methods are designed to infer individual regulatory relationships among genes from data on gene expression. We propose a novel efficient Bayesian method for discovering local causal relationships among triplets of (normally distributed) variables. In our approach, we score covariance structures for each triplet in one g…
IE University（IE大学）；Universitat de València（瓦伦西亚大学）
Causal discovery algorithms aim at untangling complex causal relationships from data. Here, we study causal discovery and inference methods based on staged tree models, which can represent complex and asymmetric causal relationships between categorical variables. We provide a first graphical representation of the equivalence class of a staged tree, by looking only at a specific subset of its underlying independences. We further define a new pre-metric, inspired by the widely used structural intervention distance, to quantify the closeness between two staged trees in terms of their correspondin…
RIKEN Center for Advanced Intelligence Project（理化学研究所先进智能项目中心）；Shiga University（滋贺大学）
Causal discovery from data affected by unobserved variables is an important but difficult problem to solve. The effects that unobserved variables have on the relationships between observed variables are more complex in nonlinear cases than in linear cases. In this study, we focus on causal additive models in the presence of unobserved variables. Causal additive models exhibit structural equations that are additive in the variables and error terms. We take into account the presence of not only unobserved common causes but also unobserved intermediate variables. Our theoretical results show that…
Causal Discovery methods aim to identify a DAG structure that represents causal relationships from observational data. In this article, we stress that it is important to test such methods for robustness in practical settings. As our main example, we analyze the NOTEARS method, for which we demonstrate a lack of scale-invariance. We show that NOTEARS is a method that aims to identify a parsimonious DAG from the data that explains the residual variance. We conclude that NOTEARS is not suitable for identifying truly causal relationships from the data.
