Despite the considerable advancements achieved by deep neural networks, their performance tends to degenerate when the test environment diverges from the training ones. Domain generalization (DG) solves this issue by learning representations independent of domain-related information, thus facilitating extrapolation to unseen environments. Existing approaches typically focus on formulating tailored training objectives to extract shared features from the source data. However, the disjointed training and testing procedures may compromise robustness, particularly in the face of unforeseen variations during deployment. In this paper, we propose a novel and holistic framework based on causality, named InPer, designed to enhance model generalization by incorporating causal intervention during training and causal perturbation during testing. Specifically, during the training phase, we employ entropy-based causal intervention (EnIn) to refine the selection of causal variables. To identify samples with anti-interference causal variables from the target domain, we propose a novel metric, homeostatic score, through causal perturbation (HoPer) to construct a prototype classifier in test time. Experimental results across multiple cross-domain tasks confirm the efficacy of InPer.
LUMLIS (Lisbon ELLIS Unit)（里斯本ELLIS单元LUMLIS）；Instituto de Telecomunicações（电信研究所）；Unbabel（Unbabel公司）
Recent work has shown promising results in causal discovery by leveraging interventional data with gradient-based methods, even when the intervened variables are unknown. However, previous work assumes that the correspondence between samples and interventions is known, which is often unrealistic. We envision a scenario with an extensive dataset sampled from multiple intervention distributions and one observation distribution, but where we do not know which distribution originated each sample and how the intervention affected the system, \textit{i.e.}, interventions are entirely latent. We prop…
The past two decades have seen a growing interest in combining causal information, commonly represented using causal graphs, with machine learning models. Probability trees provide a simple yet powerful alternative representation of causal information. They enable both computation of intervention and counterfactuals, and are strictly more general, since they allow context-dependent causal dependencies. Here we present a Bayesian method for learning probability trees from a combination of interventional and observational data. The method quantifies the expected information gain from an interven…
ETH Zürich（苏黎世联邦理工学院）；Max Planck Institute for Intelligent Systems（马克斯·普朗克智能系统研究所）；IBM Research Zürich（苏黎世IBM研究院）
Inferring causal structures from experimentation is a central task in many domains. For example, in biology, recent advances allow us to obtain single-cell expression data under multiple interventions such as drugs or gene knockouts. However, the targets of the interventions are often uncertain or unknown and the number of observations limited. As a result, standard causal discovery methods can no longer be reliably used. To fill this gap, we propose a Bayesian framework (BaCaDI) for discovering and reasoning about the causal structure that underlies data generated under various unknown experi…
University of Science and Technology of China（中国科学技术大学）；University of Hong Kong（香港大学）
Causal diagrams based on do intervention are useful tools to formalize, process and understand causal relationship among variables. However, the do intervention has controversial interpretation of causal questions for non-manipulable variables, and it also lacks the power to check the conditions related to counterfactual variables. This paper introduces a new info intervention to tackle these two problems, and provides causal diagrams for communication and theoretical focus based on this info intervention. Our info intervention intervenes the input/output information of causal mechanisms, whil…
Transforming a causal system from a given initial state to a desired target state is an important task permeating multiple fields including control theory, biology, and materials science. In causal models, such transformations can be achieved by performing a set of interventions. In this paper, we consider the problem of identifying a shift intervention that matches the desired mean of a system through active learning. We define the Markov equivalence class that is identifiable from shift interventions and propose two active learning strategies that are guaranteed to exactly match a desired me…
Adversarial training is the de facto most promising defense against adversarial examples. Yet, its passive nature inevitably prevents it from being immune to unknown attackers. To achieve a proactive defense, we need a more fundamental understanding of adversarial examples, beyond the popular bounded threat model. In this paper, we provide a causal viewpoint of adversarial vulnerability: the cause is the spurious correlation ubiquitously existing in learning, i.e., the confounding effect, where attackers are precisely exploiting these effects. Therefore, a fundamental solution for adversarial …
Latent variables often mask cause-effect relationships in observational data which provokes spurious links that may be misinterpreted as causal. This problem sparks great interest in the fields such as climate science and economics. We propose to estimate confounded causal links of time series using Sequential Causal Effect Variational Autoencoder (SCEVAE) while applying Knockoff interventions. Knockoff variables have the same distribution as the originals and preserve the correlation to other variables. This allows for counterfactuals that are more faithful to the observational distribution. …
The most fundamental problem in statistical causality is determining causal relationships from limited data. Probability trees, which combine prior causal structures with Bayesian updates, have been suggested as a possible solution. In this work, we quantify the information gain from a single intervention and show that both the anticipated information gain, prior to making an intervention, and the expected gain from an intervention have simple expressions. This results in an active-learning method that simply selects the intervention with the highest anticipated gain, which we illustrate throu…
Columbia University（哥伦比亚大学）；Rutgers University（罗格斯大学）
We introduce a framework for learning robust visual representations that generalize to new viewpoints, backgrounds, and scene contexts. Discriminative models often learn naturally occurring spurious correlations, which cause them to fail on images outside of the training distribution. In this paper, we show that we can steer generative models to manufacture interventions on features caused by confounding factors. Experiments, visualizations, and theoretical results show this method learns robust representations more consistent with the underlying causal relationships. Our approach improves per…
Deciphering interventional dynamical causality from non-intervention complex systems
Research Institute of Intelligent Complex Systems & CISOR, Fudan University（智能复杂系统研究所及CISOR，复旦大学）；International Research Center for Neurointelligence, The University of Tokyo Institutes for Advanced Study, The University of Tokyo（神经智能国际研究中心，东京大学先进研究所，东京大学）；School of Pharmacy, Shanghai University of Traditional Chinese Medicine（上海中医药大学药学院）；Institute of AI and Robotics, College of Intelligent Robotics and Advanced Manufacturing, Fudan University（人工智能与机器人研究所，智能机器人与先进制造学院，复旦大学）；Frontiers Science Center for Deep Ocean Multispheres and Earth System, Key Laboratory of Marine Chemistry Theory and Technology, Ministry of Education, Ocean University of China（深海多球体与地球系统前沿科学中心，海洋化学理论与技术重点实验室，教育部，中国海洋大学）；School of Mathematical Sciences and School of AI, Shanghai Jiao Tong University（数学科学学院和人工智能学院，上海交通大学）；Key Laboratory of Systems Health Science of Zhejiang Province, Hangzhou Institute for Advanced Study, University of Chinese Academy of Sciences, Chinese Academy of Sciences（浙江省系统健康科学重点实验室，杭州高级研究所，中国科学院大学，中国科学院）
The most widely discussed methods for estimating the Average Causal Effect/Average Treatment Effect are those for intervention in discrete binary variables whose value represents intervention/non-intervention groups. On the other hand, methods for intervening in continuous variables independent of data generating models have not been developed. In this study, we give a method for estimating the average causal effect for intervention in continuous variables that can be applied to data of any generating models as long as the causal effect is identifiable. The proposing method is independent of m…
Australian National University（澳大利亚国立大学）；Data61/NICTA（Data61/国家信息与通信技术研究院）；University of Alberta（阿尔伯塔大学）
We study the problem of using causal models to improve the rate at which good interventions can be learned online in a stochastic environment. Our formalism combines multi-arm bandits and causal inference to model a novel type of bandit feedback that is not exploited by existing approaches. We propose a new algorithm that exploits the causal feedback and prove a bound on its simple regret that is strictly better (in all quantities) than algorithms that do not use the additional causal information.
McGill University（麦吉尔大学）；ETH Zürich（苏黎世联邦理工学院）；GlaxoSmithKline（葛兰素史克）；TU Munich（慕尼黑工业大学）；Helmholtz AI（亥姆霍兹人工智能研究所）；Max Planck Institute for Intelligent Systems（马克斯·普朗克智能系统研究所）
Georgia Institute of Technology（佐治亚理工学院）；Microsoft Research India（微软印度研究院）；Microsoft（微软）；Microsoft Research AI（微软人工智能研究院）
For many kinds of interventions, such as a new advertisement, marketing intervention, or feature recommendation, it is important to target a specific subset of people for maximizing its benefits at minimum cost or potential harm. However, a key challenge is that no data is available about the effect of such a prospective intervention since it has not been deployed yet. In this work, we propose a split-treatment analysis that ranks the individuals most likely to be positively affected by a prospective intervention using past observational data. Unlike standard causal inference methods, the spli…
Relaxing Faithfulness with Intervention-Only Causal Discovery
Thayer School of Engineering Dartmouth College（达特茅斯学院塞耶工程学院）；Broad Institute of MIT and Harvard（麻省理工学院和哈佛大学布罗德研究所）；Massachusetts Institute of Technology（麻省理工学院）
University of Massachusetts Amherst（马萨诸塞大学阿默斯特分校）；Amazon（亚马逊）
We introduce a new Collaborative Causal Discovery problem, through which we model a common scenario in which we have multiple independent entities each with their own causal graph, and the goal is to simultaneously learn all these causal graphs. We study this problem without the causal sufficiency assumption, using Maximal Ancestral Graphs (MAG) to model the causal graphs, and assuming that we have the ability to actively perform independent single vertex (or atomic) interventions on the entities. If the $M$ underlying (unknown) causal graphs of the entities satisfy a natural notion of cluster…
In this paper, we discuss structure learning of causal networks from multiple data sets obtained by external intervention experiments where we do not know what variables are manipulated. For example, the conditions in these experiments are changed by changing temperature or using drugs, but we do not know what target variables are manipulated by the external interventions. From such data sets, the structure learning becomes more difficult. For this case, we first discuss the identifiability of causal structures. Next we present a graph-merging method for learning causal networks for the case t…
Trust Your $\nabla$: Gradient-based Intervention Targeting for Causal Discovery
Warsaw University（华沙大学）；IDEAS NCBR；Jagiellonian University（雅盖隆大学）；ETH Zurich（苏黎世联邦理工学院）；Helmholtz, TU Munich（慕尼黑工业大学亥姆霍兹中心）；Institute of Mathematics, Polish Academy of Sciences（波兰科学院数学研究所）
Interventions are central to causal learning and reasoning. Yet ultimately an intervention is an abstraction: an agent embedded in a physical environment (perhaps modeled as a Markov decision process) does not typically come equipped with the notion of an intervention -- its action space is typically ego-centric, without actions of the form `intervene on X'. Such a correspondence between ego-centric actions and interventions would be challenging to hard-code. It would instead be better if an agent learnt which sequence of actions allow it to make targeted manipulations of the environment, and …
Causal models bring many benefits to decision-making systems (or agents) by making them interpretable, sample-efficient, and robust to changes in the input distribution. However, spurious correlations can lead to wrong causal models and predictions. We consider the problem of inferring a causal model of a reinforcement learning environment and we propose a method to deal with spurious correlations. Specifically, our method designs a reward function that incentivizes an agent to do an intervention to find errors in the causal model. The data obtained from doing the intervention is used to impro…
Can Large Language Models Help Experimental Design for Causal Discovery?
The Hong Kong University of Science and Technology (Guangzhou)（香港科技大学（广州））；MBZUAI（穆罕默德·本·扎耶德人工智能大学）；Hong Kong Baptist University（香港浸会大学）；The University of Sydney（悉尼大学）；Carnegie Mellon University（卡内基梅隆大学）
Cause-effect analysis is crucial to understand the underlying mechanism of a system. We propose to exploit model invariance through interventions on the predictors to infer causality in nonlinear multivariate systems of time series. We model nonlinear interactions in time series using DeepAR and then expose the model to different environments using Knockoffs-based interventions to test model invariance. Knockoff samples are pairwise exchangeable, in-distribution and statistically null variables generated without knowing the response. We test model invariance where we show that the distribution…
We consider recovering a causal graph in presence of latent variables, where we seek to minimize the cost of interventions used in the recovery process. We consider two intervention cost models: (1) a linear cost model where the cost of an intervention on a subset of variables has a linear form, and (2) an identity cost model where the cost of an intervention is the same, regardless of what variables it is on, i.e., the goal is just to minimize the number of interventions. Under the linear cost model, we give an algorithm to identify the ancestral relations of the underlying causal graph, achi…
The ability to answer causal questions is crucial in many domains, as causal inference allows one to understand the impact of interventions. In many applications, only a single intervention is possible at a given time. However, in some important areas, multiple interventions are concurrently applied. Disentangling the effects of single interventions from jointly applied interventions is a challenging task -- especially as simultaneously applied interventions can interact. This problem is made harder still by unobserved confounders, which influence both treatments and outcome. We address this c…
Robustness of decision rules to shifts in the data-generating process is crucial to the successful deployment of decision-making systems. Such shifts can be viewed as interventions on a causal graph, which capture (possibly hypothetical) changes in the data-generating process, whether due to natural reasons or by the action of an adversary. We consider causal Bayesian networks and formally define the interventional robustness problem, a novel model-based notion of robustness for decision functions that measures worst-case performance with respect to a set of interventions that denote changes t…
Identifiability Guarantees for Causal Disentanglement from Soft Interventions
Broad Institute of MIT and Harvard（麻省理工学院与哈佛大学布罗德研究所）；Laboratory for Information and Decision Systems, MIT（麻省理工学院信息与决策系统实验室）；MIT-IBM Watson AI Lab（MIT-IBM沃森人工智能实验室）；IBM Research（IBM研究院）；Google Research（谷歌研究院）
Ludwig Maximilian University of Munich（慕尼黑大学）；Siemens CT（西门子中央研究院）
Estimating individual treatment effects from data of randomized experiments is a critical task in causal inference. The Stable Unit Treatment Value Assumption (SUTVA) is usually made in causal inference. However, interference can introduce bias when the assigned treatment on one unit affects the potential outcomes of the neighboring units. This interference phenomenon is known as spillover effect in economics or peer effect in social science. Usually, in randomized experiments or observational studies with interconnected units, one can only observe treatment responses under interference. Hence…
University of Oxford（牛津大学）；Max Planck Institute for Intelligent Systems（马克斯·普朗克智能系统研究所）；KTH Royal Institute of Technology（瑞典皇家理工学院）
Causal discovery from observational and interventional data is challenging due to limited data and non-identifiability: factors that introduce uncertainty in estimating the underlying structural causal model (SCM). Selecting experiments (interventions) based on the uncertainty arising from both factors can expedite the identification of the SCM. Existing methods in experimental design for causal discovery from limited data either rely on linear assumptions for the SCM or select only the intervention target. This work incorporates recent advances in Bayesian causal discovery into the Bayesian o…
University of Massachusetts（马萨诸塞大学）；Massachusetts Institute of Technology（麻省理工学院）
Causal inference can be formalized as Bayesian inference that combines a prior distribution over causal models and likelihoods that account for both observations and interventions. We show that it is possible to implement this approach using a sufficiently expressive probabilistic programming language. Priors are represented using probabilistic programs that generate source code in a domain specific language. Interventions are represented using probabilistic programs that edit this source code to modify the original generative process. This approach makes it straightforward to incorporate data…
