In many applications, there is a need to predict the effect of an intervention on different individuals from data. For example, which customers are persuadable by a product promotion? which patients should be treated with a certain type of treatment? These are typical causal questions involving the effect or the change in outcomes made by an intervention. The questions cannot be answered with traditional classification methods as they only use associations to predict outcomes. For personalised marketing, these questions are often answered with uplift modelling. The objective of uplift modelling is to estimate causal effect, but its literature does not discuss when the uplift represents causal effect. Causal heterogeneity modelling can solve the problem, but its assumption of unconfoundedness is untestable in data. So practitioners need guidelines in their applications when using the methods. In this paper, we use causal classification for a set of personalised decision making problems, and differentiate it from classification. We discuss the conditions when causal classification can be resolved by uplift (and causal heterogeneity) modelling methods. We also propose a general framework for causal classification, by using off-the-shelf supervised methods for flexible implementations. Experiments have shown two instantiations of the framework work for causal classification and for uplift (causal heterogeneity) modelling, and are competitive with the other uplift (causal heterogeneity) modelling methods.
School of Business and Economics, Humboldt-University of Berlin（柏林洪堡大学经济与商学院）；Institute of Computer Science, Polish Academy of Sciences（波兰科学院计算机科学研究所）
Uplift models support decision-making in marketing campaign planning. Estimating the causal effect of a marketing treatment, an uplift model facilitates targeting communication to responsive customers and efficient allocation of marketing budgets. Research into uplift models focuses on conversion models to maximize incremental sales. The paper introduces uplift modeling strategies for maximizing incremental revenues. If customers differ in their spending behavior, revenue maximization is a more plausible business objective compared to maximizing conversions. The proposed methodology entails a …
Uplift Modeling based on Graph Neural Network Combined with Causal Knowledge
Alipay, AntGroup（支付宝，蚂蚁集团）；Imperial College London（伦敦帝国理工学院）；University College London（伦敦大学学院）；Zhejiang University（浙江大学）；Peking University（北京大学）；Zhejiang Lab（之江实验室）
Tencent（腾讯）；Purdue University（普渡大学）；Toyota Research Institute（丰田研究院）；Spotify（声田）
Uplift modeling is a causal learning technique that estimates subgroup-level treatment effects. It is commonly used in industry and elsewhere for tasks such as targeting ads. In a typical setting, uplift models can take thousands of features as inputs, which is costly and results in problems such as overfitting and poor model interpretability. Consequently, there is a need to select a subset of the most important features for modeling. However, traditional methods for doing feature selection are not fit for the task because they are designed for standard machine learning models whose target is…
Institute of Computer Science, Polish Academy of Sciences（波兰科学院计算机科学研究所）
Uplift modeling is an area of machine learning which aims at predicting the causal effect of some action on a given individual. The action may be a medical procedure, marketing campaign, or any other circumstance controlled by the experimenter. Building an uplift model requires two training sets: the treatment group, where individuals have been subject to the action, and the control group, where no action has been performed. An uplift model allows then to assess the gain resulting from taking the action on a given individual, such as the increase in probability of patient recovery or of a prod…
Class flipping for uplift modeling and Heterogeneous Treatment Effect estimation on imbalanced RCT data
Institute of Computer Science, Polish Academy of Sciences（波兰科学院计算机科学研究所）；Faculty of Mathematics and Information Science, Warsaw University of Technology（华沙理工大学数学与信息科学学院）
Orthogonal Uplift Learning with Permutation-Invariant Representations for Combinatorial Treatments
Didi Chuxing（滴滴出行）；School of Statistics, East China Normal University（华东师范大学统计学院）；School of Mathematics and Statistics, Beijing Jiaotong University（北京交通大学数学与统计学院）；School of Statistics and Data Science, Shanghai University of Finance and Economics（上海财经大学统计与数据科学学院）；Institute of Data Science and Statistics, Shanghai University of Finance and Economics（上海财经大学数据科学与统计研究院）
Uplift is a particular case of individual treatment effect modeling. Such models deal with cause-and-effect inference for a specific factor, such as a marketing intervention. In practice, these models are built on customer data who purchased products or services to improve product marketing. Uplift is estimated using either i) conditional mean regression or ii) transformed outcome regression. Most existing approaches are adaptations of classification and regression trees for the uplift case. However, in practice, these conventional approaches are prone to overfitting. Here we propose a new met…
Criteo AI Lab（Criteo人工智能实验室）；University Grenoble Alps（格勒诺布尔阿尔卑斯大学）；ENS Paris Saclay（巴黎萨克雷高等师范学校）
Individual Treatment Effect (ITE) prediction is an important area of research in machine learning which aims at explaining and estimating the causal impact of an action at the granular level. It represents a problem of growing interest in multiple sectors of application such as healthcare, online advertising or socioeconomics. To foster research on this topic we release a publicly available collection of 13.9 million samples collected from several randomized control trials, scaling up previously available datasets by a healthy 210x factor. We provide details on the data collection and perform …
University of Melbourne（墨尔本大学）；University of Tartu（塔尔图大学）
This paper proposes an approach to analyze an event log of a business process in order to generate case-level recommendations of treatments that maximize the probability of a given outcome. Users classify the attributes in the event log into controllable and non-controllable, where the former correspond to attributes that can be altered during an execution of the process (the possible treatments). We use an action rule mining technique to identify treatments that co-occur with the outcome under some conditions. Since action rules are generated based on correlation rather than causation, we the…
Didi Chuxing（滴滴出行）；University of North Carolina at Chapel Hill（北卡罗来纳大学教堂山分校）
Uplift modeling is a rapidly growing approach that utilizes causal inference and machine learning methods to directly estimate the heterogeneous treatment effects, which has been widely applied to various online marketplaces to assist large-scale decision-making in recent years. The existing popular models, like causal forest (CF), are limited to either discrete treatments or posing parametric assumptions on the outcome-treatment relationship that may suffer model misspecification. However, continuous treatments (e.g., price, duration) often arise in marketplaces. To alleviate these restrictio…
Consequential decision-making incentivizes individuals to strategically adapt their behavior to the specifics of the decision rule. While a long line of work has viewed strategic adaptation as gaming and attempted to mitigate its effects, recent work has instead sought to design classifiers that incentivize individuals to improve a desired quality. Key to both accounts is a cost function that dictates which adaptations are rational to undertake. In this work, we develop a causal framework for strategic adaptation. Our causal perspective clearly distinguishes between gaming and improvement and …
Applying causal inference models in areas such as economics, healthcare and marketing receives great interest from the machine learning community. In particular, estimating the individual-treatment-effect (ITE) in settings such as precision medicine and targeted advertising has peaked in application. Optimising this ITE under the strong-ignorability-assumption -- meaning all confounders expressing influence on the outcome of a treatment are registered in the data -- is often referred to as uplift modeling (UM). While these techniques have proven useful in many settings, they suffer vividly in …
Harvard T.H. Chan School of Public Health（哈佛大学陈曾熙公共卫生学院）；Harvard-MIT Division of Health Sciences and Technology（哈佛-麻省理工学院健康科学与技术部）；Massachusetts General Hospital（麻省总医院）；Harvard Medical School（哈佛医学院）；Brigham and Women’s Hospital（布莱根妇女医院）
Causal inference from observational data is the goal of many data analyses in the health and social sciences. However, academic statistics has often frowned upon data analyses with a causal objective. The introduction of the term "data science" provides a historic opportunity to redefine data analysis in such a way that it naturally accommodates causal inference from observational data. Like others before, we organize the scientific contributions of data science into three classes of tasks: Description, prediction, and counterfactual prediction (which includes causal inference). An explicit cl…
Causal discovery for quantitative data has been extensively studied but less is known for categorical data. We propose a novel causal model for categorical data based on a new classification model, termed classification with optimal label permutation (COLP). By design, COLP is a parsimonious classifier, which gives rise to a provably identifiable causal model. A simple learning algorithm via comparing likelihood functions of causal and anti-causal models suffices to learn the causal direction. Through experiments with synthetic and real data, we demonstrate the favorable performance of the pro…
In this manuscript (ms), we propose causal inference based single-branch ensemble trees for uplift modeling, namely CIET. Different from standard classification methods for predictive probability modeling, CIET aims to achieve the change in the predictive probability of outcome caused by an action or a treatment. According to our CIET, two partition criteria are specifically designed to maximize the difference in outcome distribution between the treatment and control groups. Next, a novel single-branch tree is built by taking a top-down node partition approach, and the remaining samples are ce…
The foundations of cost-sensitive causal classification
Faculty of Economics and Business（经济与商业学院）；Katholieke Universiteit Leuven（鲁文大学）；Data Analytics Laboratory（数据分析实验室）；Solvay Business School（索维亚商学院）；Vrije Universiteit Brussel（布鲁塞尔自由大学）；Department of Applied Mathematics and Theoretical Physics（应用数学与理论物理系）；University of Cambridge（剑桥大学）；School of Economics and Business（经济与商业学院）；University of Chile（智利大学）
In the context of individual-level causal inference, we study the problem of predicting whether someone will respond or not to a treatment based on their features and past examples of features, treatment indicator (e.g., drug/no drug), and a binary outcome (e.g., recovery from disease). As a classification task, the problem is made difficult by not knowing the example outcomes under the opposite treatment indicators. We assume the effect is monotonic, as in advertising's effect on a purchase or bail-setting's effect on reappearance in court: either it would have happened regardless of treatmen…
Arizona State University（亚利桑那州立大学）；Northwestern Polytechnical University（西北工业大学）；Army Research Lab（陆军研究实验室）
Time series data is a collection of chronological observations which is generated by several domains such as medical and financial fields. Over the years, different tasks such as classification, forecasting, and clustering have been proposed to analyze this type of data. Time series data has been also used to study the effect of interventions over time. Moreover, in many fields of science, learning the causal structure of dynamic systems and time series data is considered an interesting task which plays an important role in scientific discoveries. Estimating the effect of an intervention and i…
This work explores the idea of a causal contextual multi-armed bandit approach to automated marketing, where we estimate and optimize the causal (incremental) effects. Focusing on causal effect leads to better return on investment (ROI) by targeting only the persuadable customers who wouldn't have taken the action organically. Our approach draws on strengths of causal inference, uplift modeling, and multi-armed bandits. It optimizes on causal treatment effects rather than pure outcome, and incorporates counterfactual generation within data collection. Following uplift modeling results, we opti…
Solvay Business School（索尔维商学院）；Vrije Universiteit Brussel（布鲁塞尔自由大学）；Faculty of Economics and Business（经济与商业学院）；Katholieke Universiteit Leuven（鲁汶大学）
Causal classification models are adopted across a variety of operational business processes to predict the effect of a treatment on a categorical business outcome of interest depending on the process instance characteristics. This allows optimizing operational decision-making and selecting the optimal treatment to apply in each specific instance, with the aim of maximizing the positive outcome rate. While various powerful approaches have been presented in the literature for learning causal classification models, no formal framework has been elaborated for optimal decision-making based on the e…
Michigan State University（密歇根州立大学）；Pacific Northwest National Laboratory（太平洋西北国家实验室）
Causal analysis and classification of injury severity applying non-parametric methods for traffic crashes has received limited attention. This study presents a methodological framework for causal inference, using Granger causality analysis, and injury severity classification of traffic crashes, occurring on interstates, with different machine learning techniques including decision trees (DT), random forest (RF), extreme gradient boosting (XGBoost), and deep neural network (DNN). The data used in this study were obtained for traffic crashes on all interstates across the state of Texas from a pe…
We discuss recent work for causal inference and predictive robustness in a unifying way. The key idea relies on a notion of probabilistic invariance or stability: it opens up new insights for formulating causality as a certain risk minimization problem with a corresponding notion of robustness. The invariance itself can be estimated from general heterogeneous or perturbation data which frequently occur with nowadays data collection. The novel methodology is potentially useful in many applications, offering more robustness and better `causal-oriented' interpretation than machine learning or est…
Inverse classification uses an induced classifier as a queryable oracle to guide test instances towards a preferred posterior class label. The result produced from the process is a set of instance-specific feature perturbations, or recommendations, that optimally improve the probability of the class label. In this work, we adopt a causal approach to inverse classification, eliciting treatment policies (i.e., feature perturbations) for models induced with causal properties. In so doing, we solve a long-standing problem of eliciting multiple, continuously valued treatment policies, using an upda…
The Role of Causal Features in Strategic Classification for Robustness and Alignment
Mila & Université de Montréal（Mila与蒙特利尔大学）；Canada CIFAR AI Chair（加拿大CIFAR人工智能主席）；Faculty of Computer Science, Technion - Israel Institute of Technology（技术学院-以色列理工学院计算机科学系）；Dept. of Computing Science, Amii & University of Alberta, Canada（计算科学系，Amii与阿尔伯塔大学，加拿大）
We Need a More Robust Classifier: Dual Causal Learning Empowers Domain-Incremental Time Series Classification
School of Software, Northeastern University（东北大学软件学院）；Institute of Artificial Intelligence and Robotics, Xi'an Jiaotong University（西安交通大学人工智能与机器人研究所）；College of Computer Science and Technology, National University of Defense Technology（国防科技大学计算机科学与技术学院）；School of Software, University of Science and Technology of China（中国科学技术大学软件学院）
Neural networks trained to classify images do so by identifying features that allow them to distinguish between classes. These sets of features are either causal or context dependent. Grad-CAM is a popular method of visualizing both sets of features. In this paper, we formalize this feature divide and provide a methodology to extract causal features from Grad-CAM. We do so by defining context features as those features that allow contrast between predicted class and any contrast class. We then apply a set theoretic approach to separate causal from contrast features for COVID-19 CT scans. We sh…
Causal and Local Correlations Based Network for Multivariate Time Series Classification
School of Information Science and Engineering, Shandong Normal University（山东师范大学信息科学与工程学院）；Shandong Provincial Key Laboratory for Distributed Computer Software Novel Technology（山东省分布式计算机软件新技术重点实验室）
Temple University（天普大学）；University of Maryland, Baltimore County（马里兰大学巴尔的摩县分校）
Text features that are correlated with class labels, but do not directly cause them, are sometimesuseful for prediction, but they may not be insightful. As an alternative to traditional correlation-basedfeature selection, causal inference could reveal more principled, meaningful relationships betweentext features and labels. To help researchers gain insight into text data, e.g. for social scienceapplications, in this paper we investigate a class of matching-based causal inference methods fortext feature selection. Features used in document classification are often high dimensional, howeverexis…
University of Science and Technology of China（中国科学技术大学）；Sea AI Lab（Sea人工智能实验室）；National University of Singapore（新加坡国立大学）
In graph classification, attention and pooling-based graph neural networks (GNNs) prevail to extract the critical features from the input graph and support the prediction. They mostly follow the paradigm of learning to attend, which maximizes the mutual information between the attended graph and the ground-truth label. However, this paradigm makes GNN classifiers recklessly absorb all the statistical correlations between input features and labels in the training data, without distinguishing the causal and noncausal effects of features. Instead of underscoring the causal features, the attended …
Are Heterogeneous Graph Neural Networks Truly Effective for Node Classification? A Causal Perspective
College of Computing and Data Science, Nanyang Technological University（计算与数据科学学院，南洋理工大学）；Joint NTU-UBC Research Centre of Excellence in Active Living for the Elderly (LILY), Nanyang Technological University（老年人积极生活卓越研究中心（LILY），南洋理工大学）；Alibaba-NTU Singapore Joint Research Institute (ANGEL), Nanyang Technological University（阿里巴巴-南洋理工大学新加坡联合研究机构（ANGEL），南洋理工大学）
Refined Causal Graph Structure Learning via Curvature for Brain Disease Classification
Institute of Innovation, Science, and Sustainability（创新、科学与可持续性研究所）；Federation University Australia（联邦大学澳大利亚分校）；Stainaz Research（斯泰纳兹研究）；MIoT & IPIN Lab（MIoT与IPIN实验室）；University of New South Wales（新南威尔士大学）；School of Information and Electronic Engineering（信息与电子工程学院）；Zhejiang Gongshang University（浙江工商大学）；School of Computing Technologies（计算技术学院）
