Many dynamic processes, including common scenarios in robotic control and reinforcement learning (RL), involve a set of interacting subprocesses. Though the subprocesses are not independent, their interactions are often sparse, and the dynamics at any given time step can often be decomposed into locally independent causal mechanisms. Such local causal structures can be leveraged to improve the sample efficiency of sequence prediction and off-policy reinforcement learning. We formalize this by introducing local causal models (LCMs), which are induced from a global causal model by conditioning on a subset of the state space. We propose an approach to inferring these structures given an object-oriented state representation, as well as a novel algorithm for Counterfactual Data Augmentation (CoDA). CoDA uses local structures and an experience replay to generate counterfactual experiences that are causally valid in the global model. We find that CoDA significantly improves the performance of RL agents in locally factored tasks, including the batch-constrained and goal-conditioned settings.
University of Toronto（多伦多大学）；Vector Institute（向量研究所）；NVIDIA（英伟达）
The number of states in a dynamic process is exponential in the number of objects, making reinforcement learning (RL) difficult in complex, multi-object domains. For agents to scale to the real world, they will need to react to and reason about unseen combinations of objects. We argue that the ability to recognize and use local factorization in transition dynamics is a key element in unlocking the power of multi-object reasoning. To this end, we show that (1) known local structure in the environment transitions is sufficient for an exponential reduction in the sample complexity of training a d…
University of Cambridge（剑桥大学）；Carnegie Mellon University（卡内基梅隆大学）；Alan Turing Institute（阿兰·图灵研究所）；Imperial College London（伦敦帝国学院）
Reinforcement learning (RL) algorithms usually require a substantial amount of interaction data and perform well only for specific tasks in a fixed environment. In some scenarios such as healthcare, however, usually only few records are available for each patient, and patients may show different responses to the same treatment, impeding the application of current RL algorithms to learn optimal policies. To address the issues of mechanism heterogeneity and related data scarcity, we propose a data-efficient RL algorithm that exploits structural causal models (SCMs) to model the state dynamics, w…
Methods for offline A/B testing and counterfactual learning are seeing rapid adoption in search and recommender systems, since they allow efficient reuse of existing log data. However, there are fundamental limits to using existing log data alone, since the counterfactual estimators that are commonly used in these methods can have large bias and large variance when the logging policy is very different from the target policy being evaluated. To overcome this limitation, we explore the question of how to design data-gathering policies that most effectively augment an existing dataset of bandit f…
Wuhan University of Science and Technology（武汉科技大学）；College of Science（理学院）；Intelligent Automobile Engineering Research Institute（智能汽车工程研究院）；School of Automobile and Traffic Engineering（汽车与交通工程学院）
Deep learning models tend not to be out-of-distribution robust primarily due to their reliance on spurious features to solve the task. Counterfactual data augmentations provide a general way of (approximately) achieving representations that are counterfactual-invariant to spurious features, a requirement for out-of-distribution (OOD) robustness. In this work, we show that counterfactual data augmentations may not achieve the desired counterfactual-invariance if the augmentation is performed by a context-guessing machine, an abstract machine that guesses the most-likely context of a given input…
School of Computer Science, University College Dublin（都柏林大学学院计算机科学学院）；Insight Centre for Data Analytics, University College Dublin（都柏林大学学院洞察数据分析中心）；VistaMilk SFI Research Centre, University College Dublin（都柏林大学学院VistaMilk SFI研究中心）
Learning from class imbalanced datasets poses challenges for many machine learning algorithms. Many real-world domains are, by definition, class imbalanced by virtue of having a majority class that naturally has many more instances than its minority class (e.g. genuine bank transactions occur much more often than fraudulent ones). Many methods have been proposed to solve the class imbalance problem, among the most popular being oversampling techniques (such as SMOTE). These methods generate synthetic instances in the minority class, to balance the dataset, performing data augmentations that im…
CoDCL: Counterfactual-Inspired Augmentation Contrastive Learning for Temporal Link Prediction in Social Networks
Jiangsu Key Laboratory of Net-Worked Collective Intelligence, School of Cyber Science and Engineering, Southeast University（江苏网络集体智能重点实验室，网络安全科学与工程学院，东南大学）；School of Mathematics, Southeast University（数学学院，东南大学）；Frontiers Science Center for Mobile Information Communication and Security, School of Mathematics, Southeast University（移动信息通信与安全前沿科学中心，数学学院，东南大学）；Purple Mountain Laboratories（紫金山实验室）
We describe Countersynth, a conditional generative model of diffeomorphic deformations that induce label-driven, biologically plausible changes in volumetric brain images. The model is intended to synthesise counterfactual training data augmentations for downstream discriminative modelling tasks where fidelity is limited by data imbalance, distributional instability, confounding, or underspecification, and exhibits inequitable performance across distinct subpopulations. Focusing on demographic attributes, we evaluate the quality of synthesized counterfactuals with voxel-based morphometry, clas…
Implicit Counterfactual Data Augmentation for Robust Learning
National Engineering Research Center for Software Engineering, Peking University, Beijing, 100871, China；Hangzhou Institute for Advanced Study, University of Chinese Academy of Sciences, Hangzhou, 310024, China；Department of Mathematics, Hong Kong Baptist University, Hong Kong, 999077, China
In online experiments where the intervention is only exposed, or "triggered", for a small subset of the population, it is critical to use variance reduction techniques to estimate treatment effects with sufficient precision to inform business decisions. Trigger-dilute analysis is often used in these situations, and reduces the sampling variance of overall intent-to-treat (ITT) effects by an order of magnitude equal to the inverse of the triggering rate; for example, a triggering rate of $5\%$ corresponds to roughly a $20x$ reduction in variance. To apply trigger-dilute analysis, one needs to k…
Tsinghua University（清华大学）；Hong Kong University of Science and Technology (Guangzhou)（香港科技大学（广州））
Graph Neural Networks (GNNs) have demonstrated great success in Knowledge Graph Completion (KGC) by modeling how entities and relations interact in recent years. However, most of them are designed to learn from the observed graph structure, which appears to have imbalanced relation distribution during the training stage. Motivated by the causal relationship among the entities on a knowledge graph, we explore this defect through a counterfactual question: "would the relation still exist if the neighborhood of entities became different from observation?". With a carefully designed instantiation …
Applied Artificial Intelligence Institute (AI), Deakin University（迪肯大学应用人工智能研究所）
Data augmentation is one of the most successful techniques to improve the classification accuracy of machine learning models in computer vision. However, applying data augmentation to tabular data is a challenging problem since it is hard to generate synthetic samples with labels. In this paper, we propose an efficient classifier with a novel data augmentation technique for tabular data. Our method called CCRAL combines causal reasoning to learn counterfactual samples for the original training samples and active learning to select useful counterfactual samples based on a region of uncertainty.…
University of Pittsburgh（匹兹堡大学）；Meta AI；University of Crete（克里特大学）
A highly accurate but overconfident model is ill-suited for deployment in critical applications such as healthcare and autonomous driving. The classification outcome should reflect a high uncertainty on ambiguous in-distribution samples that lie close to the decision boundary. The model should also refrain from making overconfident decisions on samples that lie far outside its training distribution, far-out-of-distribution (far-OOD), or on unseen samples from novel classes that lie near its training distribution (near-OOD). This paper proposes an application of counterfactual explanations in f…
Earnings call (EC), as a periodic teleconference of a publicly-traded company, has been extensively studied as an essential market indicator because of its high analytical value in corporate fundamentals. The recent emergence of deep learning techniques has shown great promise in creating automated pipelines to benefit the EC-supported financial applications. However, these methods presume all included contents to be informative without refining valuable semantics from long-text transcript and suffer from EC scarcity issue. Meanwhile, these black-box methods possess inherent difficulties in pr…
Imbalanced Graph-Level Anomaly Detection via Counterfactual Augmentation and Feature Learning
School of Cyber Science and Engineering, Wuhan University（武汉大学网络空间安全学院）；School of Computing, Macquarie University（麦考瑞大学计算学院）；Renmin Hospital of Wuhan University（武汉大学人民医院）
University of Edinburgh（爱丁堡大学）；The Alan Turing Institute（艾伦·图灵研究所）
Due to the limited availability of medical data, deep learning approaches for medical image analysis tend to generalise poorly to unseen data. Augmenting data during training with random transformations has been shown to help and became a ubiquitous technique for training neural networks. Here, we propose a novel adversarial counterfactual augmentation scheme that aims at finding the most \textit{effective} synthesised images to improve downstream tasks, given a pre-trained generative model. Specifically, we construct an adversarial game where we update the input \textit{conditional factor} of…
Counterfactual Data Augmentation with Denoising Diffusion for Graph Anomaly Detection
School of Computer and Information Engineering, Henan University（河南大学计算机与信息工程学院）；University of Electronic Science and Technology of China（电子科技大学）；Sichuan University（四川大学）；Iowa State University（爱荷华州立大学）
Causal Information Prioritization for Efficient Reinforcement Learning
National Key Laboratory for Novel Software Technology, Nanjing University（南京大学新型软件技术国家重点实验室）；University of California, San Diego（加利福尼亚大学圣地亚哥分校）；MBZUAI（穆罕默德·本·扎耶德人工智能大学）；School of Intelligence Science and Technology, Nanjing University（南京大学智能科学与技术学院）
Counterfactual Modeling with Fine-Tuned LLMs for Health Intervention Design and Sensor Data Augmentation
College of Health Solutions, Arizona State University（亚利桑那州立大学健康解决方案学院）；School of Computing and Augmented Intelligence, Arizona State University（亚利桑那州立大学计算与增强智能学院）；School of Nutritional Sciences and Wellness, University of Arizona（亚利桑那大学营养科学与健康学院）
Columbia University（哥伦比亚大学）；Microsoft Research（微软研究院）；Twitter（推特）；University of British Columbia（不列颠哥伦比亚大学）；University of Chicago（芝加哥大学）
We observe that many system policies that make threshold decisions involving a resource (e.g., time, memory, cores) naturally reveal additional, or implicit feedback. For example, if a system waits X min for an event to occur, then it automatically learns what would have happened if it waited <X min, because time has a cumulative property. This feedback tells us about alternative decisions, and can be used to improve the system policy. However, leveraging implicit feedback is difficult because it tends to be one-sided or incomplete, and may depend on the outcome of the event. As a result, exis…
In this paper, we focus on data augmentation for the extreme multi-label classification (XMC) problem. One of the most challenging issues of XMC is the long tail label distribution where even strong models suffer from insufficient supervision. To mitigate such label bias, we propose a simple and effective augmentation framework and a new state-of-the-art classifier. Our augmentation framework takes advantage of the pre-trained GPT-2 model to generate label-invariant perturbations of the input texts to augment the existing training data. As a result, it present substantial improvements over bas…
Augmenting The Weather: A Hybrid Counterfactual-SMOTE Algorithm for Improving Crop Growth Prediction When Climate Changes
College of Computer & Cyber Sciences, University of Prince Mugrin（穆格林王子大学计算机与网络科学学院）；School of Computer Science, University College Dublin（都柏林大学学院计算机科学学院）；Insight Centre for Data Analytics, University College Dublin（都柏林大学学院洞察数据分析中心）；VistaMilk SFI Research Centre, University College Dublin（都柏林大学学院VistaMilk SFI研究中心）
This study proposes a new Bayesian approach to infer binary treatment effects. The approach treats counterfactual untreated outcomes as missing observations and infers them by completing a matrix composed of realized and potential untreated outcomes using a data augmentation technique. We also develop a tailored prior that helps in the identification of parameters and induces the matrix of untreated outcomes to be approximately low rank. Posterior draws are simulated using a Markov Chain Monte Carlo sampler. While the proposed approach is similar to synthetic control methods and other related …
ETH Zurich（苏黎世联邦理工学院）；Max Planck Institute for Intelligent Systems（马克斯·普朗克智能系统研究所）；Imperial College London（帝国理工学院）；University of Cambridge（剑桥大学）
Data augmentation is commonly applied to improve performance of deep learning by enforcing the knowledge that certain transformations on the input preserve the output. Currently, the data augmentation parameters are chosen by human effort and costly cross-validation, which makes it cumbersome to apply to new datasets. We develop a convenient gradient-based method for selecting the data augmentation without validation data during training of a deep neural network. Our approach relies on phrasing data augmentation as an invariance in the prior distribution on the functions of a neural network, w…
Learning to predict missing links is important for many graph-based applications. Existing methods were designed to learn the association between observed graph structure and existence of link between a pair of nodes. However, the causal relationship between the two variables was largely ignored for learning to predict links on a graph. In this work, we visit this factor by asking a counterfactual question: "would the link still exist if the graph structure became different from observation?" Its answer, counterfactual links, will be able to augment the graph data for representation learning. …
Individualised Counterfactual Examples Using Conformal Prediction Intervals
The Alan Turing Institute（阿尔法·图灵研究院）；Department of Statistics, University of Oxford（牛津大学统计学系）；University Of Edinburgh（爱丁堡大学）；University Of Warwick（沃里克大学）；School of Mathematics and Statistics, University of Glasgow（格拉斯哥大学数学与统计学学院）
Technion - Israel Institute of Technology（以色列理工学院）
Deep learning algorithms have shown promising results in visual question answering (VQA) tasks, but a more careful look reveals that they often do not understand the rich signal they are being fed with. To understand and better measure the generalization capabilities of VQA systems, we look at their robustness to counterfactually augmented data. Our proposed augmentations are designed to make a focused intervention on a specific property of the question such that the answer changes. Using these augmentations, we propose a new robustness measure, Robustness to Augmented Data (RAD), which measur…
Distilling supervision signal from a long sequence to make predictions is a challenging task in machine learning, especially when not all elements in the input sequence contribute equally to the desired output. In this paper, we propose SpanDrop, a simple and effective data augmentation technique that helps models identify the true supervision signal in a long sequence with very few examples. By directly manipulating the input sequence, SpanDrop randomly ablates parts of the sequence at a time and ask the model to perform the same task to emulate counterfactual learning and achieve input attri…
Data-driven predictive solutions predominant in commercial applications tend to suffer from biases and stereotypes, which raises equity concerns. Prediction models may discover, use, or amplify spurious correlations based on gender or other protected personal characteristics, thus discriminating against marginalized groups. Mitigating gender bias has become an important research focus in natural language processing (NLP) and is an area where annotated corpora are available. Data augmentation reduces gender bias by adding counterfactual examples to the training dataset. In this work, we show th…
CRAwDAD: Causal Reasoning Augmentation with Dual-Agent Debate
Department of Computer Science, University of Calgary（卡莱尔大学计算机科学系）；Department of Radiology, Hotchkiss Brain Institute, and Alberta Children’s Hospital Research Institute, University of Calgary（卡莱尔大学放射学系、霍奇基斯脑研究所及阿尔伯塔儿童医院研究所在卡莱尔大学）
Offline Imitation Learning with Variational Counterfactual Reasoning
Gaoling School of Artificial Intelligence, Renmin University of China（中国人民大学高瓴人工智能学院）；Department of Computer Science, City University of Hong Kong（香港城市大学计算机科学系）；School of Engineering, Westlake University（西湖大学工学院）；DiDi Chuxing（滴滴出行）
Adversarial Label Invariant Graph Data Augmentations for Out-of-Distribution Generalization
Department of Computer Science, Purdue University（普渡大学计算机科学系）；Department of Computer Science and Engineering, The Ohio State University（俄亥俄州立大学计算机科学与工程系）；Department of Industrial and Systems Engineering, The Ohio State University（俄亥俄州立大学工业与系统工程系）
