Convex Fairness Constrained Model Using Causal Effect Estimators
The University of Tokyo（东京大学）；RIKEN（理化学研究所）
Hikaru Ogura, Akiko Takeda
备注：
10 pages, 5 figures, Accepted for the 2nd Workshop on Fairness, Accountability, Transparency, Ethics and Society on the Web (FATES on the Web 2020), held in conjunction with the WWW'20
Recent years have seen much research on fairness in machine learning. Here, mean difference (MD) or demographic parity is one of the most popular measures of fairness. However, MD quantifies not only discrimination but also explanatory bias which is the difference of outcomes justified by explanatory features. In this paper, we devise novel models, called FairCEEs, which remove discrimination while keeping explanatory bias. The models are based on estimators of causal effect utilizing propensity score analysis. We prove that FairCEEs with the squared loss theoretically outperform a naive MD constraint model. We provide an efficient algorithm for solving FairCEEs in regression and binary classification tasks. In our experiment on synthetic and real-world data in these two tasks, FairCEEs outperformed an existing model that considers explanatory bias in specific cases.
It is crucial to consider the social and ethical consequences of AI and ML based decisions for the safe and acceptable use of these emerging technologies. Fairness, in particular, guarantees that the ML decisions do not result in discrimination against individuals or minorities. Identifying and measuring reliably fairness/discrimination is better achieved using causality which considers the causal relation, beyond mere association, between the sensitive attribute (e.g. gender, race, religion, etc.) and the decision (e.g. job hiring, loan granting, etc.). The big impediment to the use of causal…
There are many different causal effect estimators in causal inference. However, it is unclear how to choose between these estimators because there is no ground-truth for causal effects. A commonly used option is to simulate synthetic data, where the ground-truth is known. However, the best causal estimators on synthetic data are unlikely to be the best causal estimators on real data. An ideal benchmark for causal estimators would both (a) yield ground-truth values of the causal effects and (b) be representative of real data. Using flexible generative models, we provide a benchmark that both yi…
Experian UK&I and EMEA DataLabs（益博睿英国及爱尔兰与欧洲、中东和非洲数据实验室）
Building machine learning models that are fair with respect to an unprivileged group is a topical problem. Modern fairness-aware algorithms often ignore causal effects and enforce fairness through modifications applicable to only a subset of machine learning models. In this work, we propose a new definition of fairness that incorporates causality through the Controlled Direct Effect (CDE). We develop regularizations to tackle classical fairness measures and present a causal regularization that satisfies our new fairness definition by removing the impact of unprivileged group variables on the m…
Pennsylvania State University（宾夕法尼亚州立大学）；Purdue University（普渡大学）
As virtually all aspects of our lives are increasingly impacted by algorithmic decision making systems, it is incumbent upon us as a society to ensure such systems do not become instruments of unfair discrimination on the basis of gender, race, ethnicity, religion, etc. We consider the problem of determining whether the decisions made by such systems are discriminatory, through the lens of causal models. We introduce two definitions of group fairness grounded in causality: fair on average causal effect (FACE), and fair on average causal effect on the treated (FACT). We use the Rubin-Neyman pot…
NTT（日本电信电话株式会社）；The University of Tokyo（东京大学）；Kyoto University（京都大学）
Machine learning is used to make decisions for individuals in various fields, which require us to achieve good prediction accuracy while ensuring fairness with respect to sensitive features (e.g., race and gender). This problem, however, remains difficult in complex real-world scenarios. To quantify unfairness under such situations, existing methods utilize {\it path-specific causal effects}. However, none of them can ensure fairness for each individual without making impractical functional assumptions on the data. In this paper, we propose a far more practical framework for learning an indivi…
In this paper we propose a causal modeling approach to intersectional fairness, and a flexible, task-specific method for computing intersectionally fair rankings. Rankings are used in many contexts, ranging from Web search results to college admissions, but causal inference for fair rankings has received limited attention. Additionally, the growing literature on causal fairness has directed little attention to intersectionality. By bringing these issues together in a formal causal framework we make the application of intersectionality in fair machine learning explicit, connected to important r…
MPI for Intelligent Systems（马克斯·普朗克智能系统研究所）；University of Cambridge（剑桥大学）；Microsoft Research（微软研究院）；University of Oxford（牛津大学）；The Alan Turing Institute（艾伦·图灵研究所）；University College London（伦敦大学学院）
Causal approaches to fairness have seen substantial recent interest, both from the machine learning community and from wider parties interested in ethical prediction algorithms. In no small part, this has been due to the fact that causal models allow one to simultaneously leverage data and expert knowledge to remove discriminatory effects from predictions. However, one of the primary assumptions in causal modeling is that you know the causal graph. This introduces a new opportunity for bias, caused by misspecifying the causal model. One common way for misspecification to occur is via unmeasure…
Harvard University（哈佛大学）；Institute for Quantitative Social Science（定量社会科学研究所）；University of Massachusetts（马萨诸塞大学）
Using the concept of principal stratification from the causal inference literature, we introduce a new notion of fairness, called principal fairness, for human and algorithmic decision-making. The key idea is that one should not discriminate among individuals who would be similarly affected by the decision. Unlike the existing statistical definitions of fairness, principal fairness explicitly accounts for the fact that individuals can be impacted by the decision. Furthermore, we explain how principal fairness differs from the existing causality-based fairness criteria. In contrast to the count…
Tandon School of Engineering New York University（纽约大学坦登工程学院）；School of Global Public Health New York University（纽约大学全球公共卫生学院）
Algorithmic systems are known to impact marginalized groups severely, and more so, if all sources of bias are not considered. While work in algorithmic fairness to-date has primarily focused on addressing discrimination due to individually linked attributes, social science research elucidates how some properties we link to individuals can be conceptualized as having causes at macro (e.g. structural) levels, and it may be important to be fair to attributes at multiple levels. For example, instead of simply considering race as a causal, protected attribute of an individual, the cause may be dist…
Estimation of causal effects is fundamental in situations were the underlying system will be subject to active interventions. Part of building a causal inference engine is defining how variables relate to each other, that is, defining the functional relationship between variables given conditional dependencies. In this paper, we deviate from the common assumption of linear relationships in causal models by making use of neural autoregressive density estimators and use them to estimate causal effects within the Pearl's do-calculus framework. Using synthetic data, we show that the approach can r…
ProPublica's analysis of recidivism predictions produced by Correctional Offender Management Profiling for Alternative Sanctions (COMPAS) software tool for the task, has shown that the predictions were racially biased against African American defendants. We analyze the COMPAS data using a causal reformulation of the underlying algorithmic fairness problem. Specifically, we assess whether COMPAS exhibits racial bias against African American defendants using FACT, a recently introduced causality grounded measure of algorithmic fairness. We use the Neyman-Rubin potential outcomes framework for ca…
Simulators make unique benchmarks for causal effect estimation since they do not rely on unverifiable assumptions or the ability to intervene on real-world systems, but are often too simple to capture important aspects of real applications. We propose a simulator of Alzheimer's disease aimed at modeling intricacies of healthcare data while enabling benchmarking of causal effect and policy estimators. We fit the system to the Alzheimer's Disease Neuroimaging Initiative (ADNI) dataset and ground hand-crafted components in results from comparative treatment trials and observational treatment patt…
Arizona State University（亚利桑那州立大学）；City University of Hong Kong（香港城市大学）
An important problem in causal inference is to break down the total effect of a treatment on an outcome into different causal pathways and to quantify the causal effect in each pathway. For instance, in causal fairness, the total effect of being a male employee (i.e., treatment) constitutes its direct effect on annual income (i.e., outcome) and the indirect effect via the employee's occupation (i.e., mediator). Causal mediation analysis (CMA) is a formal statistical framework commonly used to reveal such underlying causal mechanisms. One major challenge of CMA in observational studies is handl…
Kalisch and Bühlmann (2007) showed that for linear Gaussian models, under the Causal Markov Assumption, the Strong Causal Faithfulness Assumption, and the assumption of causal sufficiency, the PC algorithm is a uniformly consistent estimator of the Markov Equivalence Class of the true causal DAG for linear Gaussian models; it follows from this that for the identifiable causal effects in the Markov Equivalence Class, there are uniformly consistent estimators of causal effects as well. The $k$-Triangle-Faithfulness Assumption is a strictly weaker assumption that avoids some implausible implicati…
What is the ideal regression (if any) for estimating average causal effects? We study this question in the setting of discrete covariates, deriving expressions for the finite-sample variance of various stratification estimators. This approach clarifies the fundamental statistical phenomena underlying many widely-cited results. Our exposition combines insights from three distinct methodological traditions for studying causal effect estimation: potential outcomes, causal diagrams, and structural models with additive errors.
Local Causal Discovery for Structural Evidence of Direct Discrimination
Cornell Tech（康奈尔理工学院）；Stevens Institute of Technology（史蒂文斯理工学院）；University of Oxford（牛津大学）；Amazon AWS AI/ML（亚马逊云科技人工智能/机器学习部门）；Weill Cornell Medicine（威尔康奈尔医学院）
University of Hong Kong（香港大学）；Shanghai Jiao Tong University（上海交通大学）；Shanghai Artificial Intelligence Laboratory（上海人工智能实验室）；Columbia University（哥伦比亚大学）
Causal mediation analysis can unpack the black box of causality and is therefore a powerful tool for disentangling causal pathways in biomedical and social sciences, and also for evaluating machine learning fairness. To reduce bias for estimating Natural Direct and Indirect Effects in mediation analysis, we propose a new method called DeepMed that uses deep neural networks (DNNs) to cross-fit the infinite-dimensional nuisance functions in the efficient influence functions. We obtain novel theoretical results that our DeepMed method (1) can achieve semiparametric efficiency bound without imposi…
Modern causal inference methods allow machine learning to be used to weaken parametric modeling assumptions. However, the use of machine learning may result in complications for inference. Doubly-robust cross-fit estimators have been proposed to yield better statistical properties. We conducted a simulation study to assess the performance of several different estimators for the average causal effect (ACE). The data generating mechanisms for the simulated treatment and outcome included log-transforms, polynomial terms, and discontinuities. We compared singly-robust estimators (g-computation, in…
We propose an improved estimator of the complier average causal effect (CACE). Researchers typically choose a presumably-unbiased estimator for the CACE in studies with noncompliance, when many other lower-variance estimators may be available. We propose a synthetic estimator that combines information across all available estimators, leveraging the efficiency in lower-variance estimators while maintaining low bias. Our approach minimizes an estimate of the mean squared error of all convex combinations of the candidate estimators. We derive the asymptotic distribution of the synthetic estimator…
Spirtes, Glymour and Scheines [Causation, Prediction, and Search (1993) Springer] described a pointwise consistent estimator of the Markov equivalence class of any causal structure that can be represented by a directed acyclic graph for any parametric family with a uniformly consistent test of conditional independence, under the Causal Markov and Causal Faithfulness assumptions. Robins et al. [Biometrika 90 (2003) 491-515], however, proved that there are no uniformly consistent estimators of Markov equivalence classes of causal structures under those assumptions. Subsequently, Kalisch and Bühl…
This paper introduces a simple framework of counterfactual estimation for causal inference with time-series cross-sectional data, in which we estimate the average treatment effect on the treated by directly imputing counterfactual outcomes for treated observations. We discuss several novel estimators under this framework, including the fixed effects counterfactual estimator, interactive fixed effects counterfactual estimator, and matrix completion estimator. They provide more reliable causal estimates than conventional twoway fixed effects models when treatment effects are heterogeneous or uno…
Regression adjustments are often considered by investigators to improve the estimation efficiency of causal effect in randomized experiments when there exists many pre-experiment covariates. In this paper, we provide conditions that guarantee the penalized regression including the Ridge, Elastic Net and Adapive Lasso adjusted causal effect estimators are asymptotic normal and we show that their asymptotic variances are no greater than that of the simple difference-in-means estimator, as long as the penalized estimators are risk consistent. We also provide conservative estimators for the asympt…
Kernel-based estimators for functional causal effects
School of Mathematical Sciences, Horizon Digital Economy Institute, University of Nottingham, Nottingham, UK（数学科学学院，Horizon数字经济研究所，诺丁汉大学，诺丁汉，英国）；Department of Statistics, Rice University, USA（统计学系，里士满大学，美国；伯克利国家实验室，美国）；Lawrence Berkeley National Laboratory, USA（统计科学组，洛斯阿拉莫斯国家实验室，美国）；Statistical Sciences Group, Los Alamos National Laboratory, USA（数学科学学院，诺丁汉大学，诺丁汉，英国）；School of Mathematical Sciences, University of Nottingham, Nottingham, UK
We propose a novel framework for matching estimators for causal effect from observational data that is based on minimizing the dual norm of estimation error when expressed as an operator. We show that many popular matching estimators can be expressed as optimal in this framework, including nearest-neighbor matching, coarsened exact matching, and mean-matched sampling. This reveals their motivation and aptness as structural priors formulated by embedding the effect in a particular functional space. This also gives rise to a range of new, kernel-based matching estimators that arise when one embe…
Relationship between two popular modeling frameworks of causal inference from observational data, namely, causal graphical model and potential outcome causal model is discussed. How some popular causal effect estimators found in applications of the potential outcome causal model, such as inverse probability of treatment weighted estimator and doubly robust estimator can be obtained by using the causal graphical model is shown. We confine to the simple case of binary outcome and treatment variables with discrete confounders and it is shown how to generalize results to cases of continuous variab…
With the increasing pervasive use of machine learning in social and economic settings, there has been an interest in the notion of machine bias in the AI community. Models trained on historic data reflect biases that exist in society and propagated them to the future through their decisions. There are three prominent metrics of machine fairness used in the community, and it has been shown statistically that it is impossible to satisfy them all at the same time. This has led to an ambiguity with regards to the definition of fairness. In this report, a causal perspective to the impossibility the…
The instrumental variable method consistently estimates the effect of a treatment when there is unmeasured confounding and a valid instrumental variable. A valid instrumental variable is a variable that is independent of unmeasured confounders and affects the treatment but does not have a direct effect on the outcome beyond its effect on the treatment. Two commonly used estimators for using an instrumental variable to estimate a treatment effect are the two stage least squares estimator and the control function estimator. For linear causal effect models, these two estimators are equivalent, bu…
Adjusting for covariates is a well established method to estimate the total causal effect of an exposure variable on an outcome of interest. Depending on the causal structure of the mechanism under study there may be different adjustment sets, equally valid from a theoretical perspective, leading to identical causal effects. However, in practice, with finite data, estimators built on different sets may display different precision. To investigate the extent of this variability we consider the simplest non-trivial non-linear model of a v-structure on three nodes for binary data. We explicitly co…
There has been increasing interest in recent years in the development of approaches to estimate causal effects when the number of potential confounders is prohibitively large. This growth in interest has led to a number of potential estimators one could use in this setting. Each of these estimators has different operating characteristics, and it is unlikely that one estimator will outperform all others across all possible scenarios. Coupling this with the fact that an analyst can never know which approach is best for their particular data, we propose a synthetic estimator that averages over a …
Weighting methods are used in observational studies to adjust for covariate imbalances between treatment and control groups. Entropy balancing (EB) is an alternative to inverse probability weighting with an estimated propensity score. The EB weights are constructed to satisfy balance constraints and optimized towards stability. We describe large sample properties of EB estimators of the average causal treatment effect, based on the Kullback-Leibler and quadratic Rényi relative entropies. Additionally, we propose estimators of their asymptotic variances. Even though the objective of EB is to re…
Under the potential outcomes framework, we introduce matched-pair factorial designs, and propose the matched-pair estimator of the factorial effects. We also calculate the randomization-based covariance matrix of the matched-pair estimator, and provide the "Neymanian" estimator of the covariance matrix.
University of Arkansas（阿肯色大学）；University of Illinois at Urbana-Champaign（伊利诺伊大学厄巴纳-香槟分校）
A recent trend of fair machine learning is to define fairness as causality-based notions which concern the causal connection between protected attributes and decisions. However, one common challenge of all causality-based fairness notions is identifiability, i.e., whether they can be uniquely measured from observational data, which is a critical barrier to applying these notions to real-world situations. In this paper, we develop a framework for measuring different causality-based fairness. We propose a unified definition that covers most of previous causality-based fairness notions, namely th…
Matching estimators for average treatment effects are widely used in the binary treatment setting, in which missing potential outcomes are imputed as the average of observed outcomes of all matches for each unit. With more than two treatment groups, however, estimation using matching requires additional techniques. In this paper, we propose a nearest-neighbors matching estimator for use with multiple, nominal treatments, and use simulations to show that this method is precise and has coverage levels that are close to nominal. In addition, we implement the proposed inference methods to examine …
Causal inference concerns not only the average effect of the treatment on the outcome but also the underlying mechanism through an intermediate variable of interest. Principal stratification characterizes such a mechanism by targeting subgroup causal effects within principal strata, which are defined by the joint potential values of an intermediate variable. Due to the fundamental problem of causal inference, principal strata are inherently latent, rendering it challenging to identify and estimate subgroup effects within them. A line of research leverages the principal ignorability assumption …
The era of big data has witnessed an increasing availability of multiple data sources for statistical analyses. We consider estimation of causal effects combining big main data with unmeasured confounders and smaller validation data with supplementary information on these confounders. Under the unconfoundedness assumption with completely observed confounders, the smaller validation data allow for constructing consistent estimators for causal effects, but the big main data can only give error-prone estimators in general. However, by leveraging the information in the big main data in a principle…
The population-attributable fraction (PAF) is a popular epidemiological measure for the burden of a harmful exposure within a population. It is often interpreted causally as proportion of preventable cases after an elimination of exposure. Originally, the PAF has been defined for cohort studies of fixed length with a baseline exposure or cross-sectional studies. An extension of the definition to complex time-to-event data is not straightforward. We revise the proposed approaches in literature and provide a clear concept of the PAF for these data situations. The conceptualization is achieved by…
Split-plot designs find wide applicability in multifactor experiments with randomization restrictions. Practical considerations often warrant the use of unbalanced designs. This paper investigates randomization based causal inference in split-plot designs that are possibly unbalanced. Extension of ideas from the recently studied balanced case yields an expression for the sampling variance of a treatment contrast estimator as well as a conservative estimator of the sampling variance. However, the bias of this variance estimator does not vanish even when the treatment effects are strictly additi…
Intercurrent events (ICEs) and missing values are inevitable in clinical trials of any size and duration, making it difficult to assess the treatment effect for all patients in randomized clinical trials. Defining the appropriate estimand that is relevant to the clinical research question is the first step in analyzing data. The tripartite estimands, which evaluate the treatment differences in the proportion of patients with ICEs due to adverse events, the proportion of patients with ICEs due to lack of efficacy, and the primary efficacy outcome for those who can adhere to study treatment unde…
