$χ$SPN: Characteristic Interventional Sum-Product Networks for Causal Inference in Hybrid Domains
Indian Institute of Technology Bombay(印度孟买理工学院);Technical University of Darmstadt(达姆施塔特工业大学);Hessian Center for Artificial Intelligence (hessian.AI)(黑森人工智能中心);German Research Center for Artificial Intelligence (DFKI)(德国人工智能研究中心);Eindhoven University of Technology(埃因霍温理工大学)
Harsh Poonia, Moritz Willig, Zhongjie Yu, Matej Zečević, Kristian Kersting, Devendra Singh Dhami
备注:
17 pages, 11 figures. Accepted as poster at UAI (Uncertainty in Artificial Intelligence) 2024
Causal inference in hybrid domains, characterized by a mixture of discrete and continuous variables, presents a formidable challenge. We take a step towards this direction and propose Characteristic Interventional Sum-Product Network ($χ$SPN) that is capable of estimating interventional distributions in presence of random variables drawn from mixed distributions. $χ$SPN uses characteristic functions in the leaves of an interventional SPN (iSPN) thereby providing a unified view for discrete and continuous random variables through the Fourier-Stieltjes transform of the probability measures. A neural network is used to estimate the parameters of the learned iSPN using the intervened data. Our experiments on 3 synthetic heterogeneous datasets suggest that $χ$SPN can effectively capture the interventional distributions for both discrete and continuous variables while being expressive and causally adequate. We also show that $χ$SPN generalize to multiple interventions while being trained only on a single intervention data.
TU Darmstadt(达姆施塔特工业大学);The University of Texas at Dallas(德克萨斯大学达拉斯分校);Hessian Center for AI (hessian.AI)(黑森人工智能中心 (hessian.AI))
While probabilistic models are an important tool for studying causality, doing so suffers from the intractability of inference. As a step towards tractable causal models, we consider the problem of learning interventional distributions using sum-product networks (SPNs) that are over-parameterized by gate functions, e.g., neural networks. Providing an arbitrarily intervened causal graph as input, effectively subsuming Pearl's do-operator, the gate function predicts the parameters of the SPN. The resulting interventional SPNs are motivated and illustrated by a structural causal model themed arou…
CausalChapter: Improving Long-Video Chaptering with Interventional Dependency Modeling
School of Artificial Intelligence, Beijing Normal University(北京师范大学人工智能学院);Beijing Key Laboratory of Artificial Intelligence for Education(北京市教育人工智能重点实验室);Engineering Research Center of Intelligent Technology and Educational Application, Ministry of Education(教育部智能技术与教育应用工程研究中心)
In this paper, we study the problem of discovering the Markov blanket (MB) of a target variable from multiple interventional datasets. Datasets attained from interventional experiments contain richer causal information than passively observed data (observational data) for MB discovery. However, almost all existing MB discovery methods are designed for finding MBs from a single observational dataset. To identify MBs from multiple interventional datasets, we face two challenges: (1) unknown intervention variables; (2) nonidentical data distributions. To tackle the challenges, we theoretically an…
Incorporating causal knowledge and mechanisms is essential for refining causal models and improving downstream tasks such as designing new treatments. In this paper, we introduce a novel concept in causal discovery, termed interventional constraints, which differs fundamentally from interventional data. While interventional data require direct perturbations of variables, interventional constraints encode high-level causal knowledge in the form of inequality constraints on causal effects. For instance, in the Sachs dataset (Sachs et al.\ 2005), Akt has been shown to be activated by PIP3, meanin…
We explore how observational and interventional causal discovery methods can be combined. A state-of-the-art observational causal discovery algorithm for time series capable of handling latent confounders and contemporaneous effects, called LPCMCI, is extended to profit from casual constraints found through randomized control trials. Numerical results show that, given perfect interventional constraints, the reconstructed structural causal models (SCMs) of the extended LPCMCI allow 84.6% of the time for the optimal prediction of the target variable. The implementation of interventional and obse…
Max Planck Institute for Intelligent Systems(马克斯·普朗克智能系统研究所);University of Cambridge(剑桥大学);National University of Singapore(新加坡国立大学)
Modern machine learning approaches excel in static settings where a large amount of i.i.d. training data are available for a given task. In a dynamic environment, though, an intelligent agent needs to be able to transfer knowledge and re-use learned components across domains. It has been argued that this may be possible through causal models, aiming to mirror the modularity of the real world in terms of independent causal mechanisms. However, the true causal structure underlying a given set of data is generally not identifiable, so it is desirable to have means to quantify differences between …
Unobserved confounding is one of the main challenges when estimating causal effects. We propose a causal reduction method that, given a causal model, replaces an arbitrary number of possibly high-dimensional latent confounders with a single latent confounder that takes values in the same space as the treatment variable, without changing the observational and interventional distributions the causal model entails. This allows us to estimate the causal effect in a principled way from combined data without relying on the common but often unrealistic assumption that all confounders have been observ…
Causal inference is central to many areas of artificial intelligence, including complex reasoning, planning, knowledge-base construction, robotics, explanation, and fairness. An active community of researchers develops and enhances algorithms that learn causal models from data, and this work has produced a series of impressive technical advances. However, evaluation techniques for causal modeling algorithms have remained somewhat primitive, limiting what we can learn from experimental studies of algorithm performance, constraining the types of algorithms and model representations that research…
Arizona State University(亚利桑那州立大学);Army Research Lab(陆军研究实验室)
We propose a generative Causal Adversarial Network (CAN) for learning and sampling from conditional and interventional distributions. In contrast to the existing CausalGAN which requires the causal graph to be given, our proposed framework learns the causal relations from the data and generates samples accordingly. The proposed CAN comprises a two-fold process namely Label Generation Network (LGN) and Conditional Image Generation Network (CIGN). The LGN is a GAN-based architecture which learns and samples from the causal model over labels. The sampled labels are then fed to CIGN, a conditional…
PACER: Acyclic Causal Discovery from Large-Scale Interventional Data
Swiss Federal Technology Institute of Lausanne (EPFL), Switzerland(瑞士联邦理工学院洛桑分校);Cornell University, USA(康奈尔大学);ETH Zurich, Zurich, Switzerland(苏黎世联邦理工学院)
University of Illinois at Urbana-Champaign(伊利诺伊大学厄巴纳-香槟分校);Sharif University of Technology(谢里夫理工大学);École Polytechnique Fédérale de Lausanne(洛桑联邦理工学院)
It is known that from purely observational data, a causal DAG is identifiable only up to its Markov equivalence class, and for many ground truth DAGs, the direction of a large portion of the edges will be remained unidentified. The golden standard for learning the causal DAG beyond Markov equivalence is to perform a sequence of interventions in the system and use the data gathered from the interventional distributions. We consider a setup in which given a budget $k$, we design $k$ interventions non-adaptively. We cast the problem of finding the best intervention target set as an optimization p…
ACTIVA: Amortized Causal Effect Estimation via Variational Autoencoders
Learning and Reasoning Group(学习与推理组);Vrije Universiteit Amsterdam(阿姆斯特丹自由大学);Leiden Institute of Advanced Computer Science (LIACS)(莱顿先进计算机科学研究所);Leiden University(莱顿大学);Informatics Institute & Institute for Logic, Language and Computation(信息研究所及逻辑、语言与计算研究所);University of Amsterdam(阿姆斯特丹大学)
Large-Scale Bayesian Causal Discovery with Interventional Data
Department of Computer and Information Science, University of Pennsylvania(计算机与信息科学系,宾夕法尼亚大学);Genomics and Computational Biology Graduate Group, University of Pennsylvania(基因组与计算生物学研究生组,宾夕法尼亚大学);Division of Informatics, University of Pennsylvania(信息学 division,宾夕法尼亚大学);Department of Genetics, University of Pennsylvania(遗传学系,宾夕法尼亚大学)
Polytechnique Montréal(蒙特利尔理工学院);IRT Saint Exupéry Canada(加拿大圣埃克苏佩里技术研究所);Ubisoft La Forge(育碧La Forge实验室);Inria Bordeaux Sud-Ouest(法国国家信息与自动化研究所波尔多西南分部)
Learning efficiently a causal model of the environment is a key challenge of model-based RL agents operating in POMDPs. We consider here a scenario where the learning agent has the ability to collect online experiences through direct interactions with the environment (interventional data), but has also access to a large collection of offline experiences, obtained by observing another agent interacting with the environment (observational data). A key ingredient, that makes this situation non-trivial, is that we allow the observed agent to interact with the environment based on hidden informatio…
Robustness of decision rules to shifts in the data-generating process is crucial to the successful deployment of decision-making systems. Such shifts can be viewed as interventions on a causal graph, which capture (possibly hypothetical) changes in the data-generating process, whether due to natural reasons or by the action of an adversary. We consider causal Bayesian networks and formally define the interventional robustness problem, a novel model-based notion of robustness for decision functions that measures worst-case performance with respect to a set of interventions that denote changes t…
ETH Zurich(苏黎世联邦理工学院);MPI for Intelligent Systems(马克斯·普朗克智能系统研究所)
The ability to learn disentangled representations that split underlying sources of variation in high dimensional, unstructured data is important for data efficient and robust use of neural networks. While various approaches aiming towards this goal have been proposed in recent times, a commonly accepted definition and validation procedure is missing. We provide a causal perspective on representation learning which covers disentanglement and domain shift robustness as special cases. Our causal framework allows us to introduce a new metric for the quantitative evaluation of deep latent variable …
Mila, Université de Montréal(蒙特利尔大学Mila);Element AI;Canada CIFAR AI Chair(加拿大CIFAR人工智能主席)
Learning a causal directed acyclic graph from data is a challenging task that involves solving a combinatorial problem for which the solution is not always identifiable. A new line of work reformulates this problem as a continuous constrained optimization one, which is solved via the augmented Lagrangian method. However, most methods based on this idea do not make use of interventional data, which can significantly alleviate identifiability issues. This work constitutes a new step in this direction by proposing a theoretically-grounded method based on neural networks that can leverage interven…
Interventional Fairness on Partially Known Causal Graphs: A Constrained Optimization Approach
School of Mathematics and Statistics, The University of Melbourne(墨尔本大学数学与统计学院);School of Data Science, Fudan University(复旦大学数据科学学院);Mohamed bin Zayed University of Artificial Intelligence(穆罕默德·本·扎耶德人工智能大学)
Causal Interventional Prediction System for Robust and Explainable Effect Forecasting
Zhejiang University(浙江大学);Hangzhou High-Tech Zone (Binjiang) Institute of Blockchain and Data Security(杭州高新区(滨江)区块链与数据安全研究院);Ant Group(蚂蚁集团);Nanyang Technological University(南洋理工大学)
Max Planck Institute for Intelligent Systems(马克斯·普朗克智能系统研究所);Saarland University(萨尔大学)
In this paper, we introduce VACA, a novel class of variational graph autoencoders for causal inference in the absence of hidden confounders, when only observational data and the causal graph are available. Without making any parametric assumptions, VACA mimics the necessary properties of a Structural Causal Model (SCM) to provide a flexible and practical framework for approximating interventions (do-operator) and abduction-action-prediction steps. As a result, and as shown by our empirical results, VACA accurately approximates the interventional and counterfactual distributions on diverse SCMs…
IBM Research(IBM研究院);MIT-IBM Watson AI Lab(麻省理工学院-IBM沃森人工智能实验室);MIT(麻省理工学院)
Directed acyclic graph (DAG) models are popular for capturing causal relationships. From observational and interventional data, a DAG model can only be determined up to its \emph{interventional Markov equivalence class} (I-MEC). We investigate the size of MECs for random DAG models generated by uniformly sampling and ordering an Erdős-Rényi graph. For constant density, we show that the expected $\log$ observational MEC size asymptotically (in the number of vertices) approaches a constant. We characterize I-MEC size in a similar fashion in the above settings with high precision. We show that th…
Estimating Joint Interventional Distributions from Marginal Interventional Data
Max Planck Institute for Intelligent Systems Tübingen, Germany(图宾根马克斯·普朗克智能系统研究所,德国);Amazon Tübingen, Germany(图宾根亚马逊公司,德国);Max Planck Institute for Intelligent Systems ELLIS Institute Tübingen, Germany(图宾根马克斯·普朗克智能系统研究所ELLIS研究所,德国)
We uncover an ever-overlooked deficiency in the prevailing Few-Shot Learning (FSL) methods: the pre-trained knowledge is indeed a confounder that limits the performance. This finding is rooted from our causal assumption: a Structural Causal Model (SCM) for the causalities among the pre-trained knowledge, sample features, and labels. Thanks to it, we propose a novel FSL paradigm: Interventional Few-Shot Learning (IFSL). Specifically, we develop three effective IFSL algorithmic implementations based on the backdoor adjustment, which is essentially a causal intervention towards the SCM of many-sh…
DoFlow: Flow-based Generative Models for Interventional and Counterfactual Forecasting on Time Series
H. Milton Stewart School of Industrial and Systems Engineering(H. Milton Stewart工业与系统工程学院);Georgia Institute of Technology(佐治亚理工学院);Northwestern–Argonne Institute for Scientific and Engineering Excellence(西北-阿贡科学与工程卓越研究所)
Deciphering interventional dynamical causality from non-intervention complex systems
Research Institute of Intelligent Complex Systems & CISOR, Fudan University(智能复杂系统研究所及CISOR,复旦大学);International Research Center for Neurointelligence, The University of Tokyo Institutes for Advanced Study, The University of Tokyo(神经智能国际研究中心,东京大学先进研究所,东京大学);School of Pharmacy, Shanghai University of Traditional Chinese Medicine(上海中医药大学药学院);Institute of AI and Robotics, College of Intelligent Robotics and Advanced Manufacturing, Fudan University(人工智能与机器人研究所,智能机器人与先进制造学院,复旦大学);Frontiers Science Center for Deep Ocean Multispheres and Earth System, Key Laboratory of Marine Chemistry Theory and Technology, Ministry of Education, Ocean University of China(深海多球体与地球系统前沿科学中心,海洋化学理论与技术重点实验室,教育部,中国海洋大学);School of Mathematical Sciences and School of AI, Shanghai Jiao Tong University(数学科学学院和人工智能学院,上海交通大学);Key Laboratory of Systems Health Science of Zhejiang Province, Hangzhou Institute for Advanced Study, University of Chinese Academy of Sciences, Chinese Academy of Sciences(浙江省系统健康科学重点实验室,杭州高级研究所,中国科学院大学,中国科学院)