The objective of this article is to introduce a fairness interpretability framework for measuring and explaining the bias in classification and regression models at the level of a distribution. In our work, we measure the model bias across sub-population distributions in the model output using the Wasserstein metric. To properly quantify the contributions of predictors, we take into account the favorability of both the model and predictors with respect to the non-protected class. The quantification is accomplished by the use of transport theory, which gives rise to the decomposition of the model bias and bias explanations to positive and negative contributions. To gain more insight into the role of favorability and allow for additivity of bias explanations, we adapt techniques from cooperative game theory.
This article is a companion paper to our earlier work Miroshnikov et al. (2021) on fairness interpretability, which introduces bias explanations. In the current work, we propose a bias mitigation methodology based upon the construction of post-processed models with fairer regressor distributions for Wasserstein-based fairness metrics. By identifying the list of predictors contributing the most to the bias, we reduce the dimensionality of the problem by mitigating the bias originating from those predictors. The post-processing methodology involves reshaping the predictor distributions by balanc…
From Fragile to Certified: Wasserstein Audits of Group Fairness Under Distribution Shift
Max Planck Institute for Intelligent Systems（马克斯·普朗克智能系统研究所）；Tübingen AI Center（图宾根人工智能中心）；Mila Québec AI Institute（魁北克人工智能研究所）；McGill University（麦吉尔大学）
Wasserstein projection distance for fairness testing of regression models
Department of Computer Science, University of British Columbia（计算机科学系，不列颠哥伦比亚大学）；Department of Pathology and Laboratory Medicine, University of British Columbia（病理学与实验室医学系，不列颠哥伦比亚大学）；Department of Statistics, University of British Columbia（统计学系，不列颠哥伦比亚大学）；BC Cancer Research, Part of Provincial Health Care Authority（BC癌症研究，省卫生保健机构的一部分）；Department of Mathematics, University of British Columbia（数学系，不列颠哥伦比亚大学）
University of Texas at Austin（德克萨斯大学奥斯汀分校）；Stanford University（斯坦福大学）
We propose a distributionally robust classification model with a fairness constraint that encourages the classifier to be fair in view of the equality of opportunity criterion. We use a type-$\infty$ Wasserstein ambiguity set centered at the empirical distribution to model distributional uncertainty and derive a conservative reformulation for the worst-case equal opportunity unfairness measure. We establish that the model is equivalent to a mixed binary optimization problem, which can be solved by standard off-the-shelf solvers. To improve scalability, we further propose a convex, hinge-loss-b…
Wasserstein Distributionally Robust Optimization Through the Lens of Structural Causal Models and Individual Fairness
Max Planck Institute for Intelligent Systems（马克斯·普朗克智能系统研究所）；Tübingen AI Center（图宾根人工智能中心）；Mila Québec AI Institute（魁北克人工智能研究所）；McGill University（麦吉尔大学）
Criminal recidivism models are tools that have gained widespread adoption by parole boards across the United States to assist with parole decisions. These models take in large amounts of data about an individual and then predict whether an individual would commit a crime if released on parole. Although such models are not the only or primary factor in making the final parole decision, questions have been raised about their accuracy, fairness, and interpretability. In this paper, various machine learning-based criminal recidivism models are created based on a real-world parole decision dataset …
We propose an approach to fair classification that enforces independence between the classifier outputs and sensitive information by minimizing Wasserstein-1 distances. The approach has desirable theoretical properties and is robust to specific choices of the threshold used to obtain class predictions from model outputs. We introduce different methods that enable hiding sensitive information at test time or have a simple and fast implementation. We show empirical performance against different fairness baselines on several benchmark fairness datasets.
The recent release of large-scale healthcare datasets has greatly propelled the research of data-driven deep learning models for healthcare applications. However, due to the nature of such deep black-boxed models, concerns about interpretability, fairness, and biases in healthcare scenarios where human lives are at stake call for a careful and thorough examinations of both datasets and models. In this work, we focus on MIMIC-IV (Medical Information Mart for Intensive Care, version IV), the largest publicly available healthcare dataset, and conduct comprehensive analyses of dataset representati…
University of Texas at Austin（德克萨斯大学奥斯汀分校）；CognitiveScale
As artificial intelligence plays an increasingly important role in our society, there are ethical and moral obligations for both businesses and researchers to ensure that their machine learning models are designed, deployed, and maintained responsibly. These models need to be rigorously audited for fairness, robustness, transparency, and interpretability. A variety of methods have been developed that focus on these issues in isolation, however, managing these methods in conjunction with model development can be cumbersome and timeconsuming. In this paper, we introduce a unified and model-agnos…
Université Paris-Saclay（巴黎萨克雷大学）；CNRS（法国国家科学研究中心）；Inria（法国国家信息与自动化研究所）；Université Gustave Eiffel（古斯塔夫·埃菲尔大学）；University of Genoa（热那亚大学）；Istituto Italiano di Tecnologia（意大利技术研究院）；University College London（伦敦大学学院）
We study the problem of learning a real-valued function that satisfies the Demographic Parity constraint. It demands the distribution of the predicted output to be independent of the sensitive attribute. We consider the case that the sensitive attribute is available for prediction. We establish a connection between fair regression and optimal transport theory, based on which we derive a close form expression for the optimal fair predictor. Specifically, we show that the distribution of this optimum is the Wasserstein barycenter of the distributions induced by the standard regression function o…
College of Computer Science, Zhejiang University（浙江大学计算机学院）；Polytechnic Institute, Zhejiang University（浙江大学理工学院）；School of Communication Engineering, Hangzhou Dianzi University（杭州电子科技大学通信工程学院）
Harvard John A. Paulson School of Engineering and Applied Sciences（哈佛大学约翰·A·保尔森工程与应用科学学院）；Radcliffe Institute for Advanced Study（拉德克利夫高等研究院）；Microsoft Research（微软研究院）；Weizmann Institute of Science（魏茨曼科学研究所）
It is well understood that classification algorithms, for example, for deciding on loan applications, cannot be evaluated for fairness without taking context into account. We examine what can be learned from a fairness oracle equipped with an underlying understanding of ``true'' fairness. The oracle takes as input a (context, classifier) pair satisfying an arbitrary fairness definition, and accepts or rejects the pair according to whether the classifier satisfies the underlying fairness truth. Our principal conceptual result is an extraction procedure that learns the underlying truth; moreover…
Escola Politécnica da Universidade de São Paulo, USP（圣保罗大学理工学院，USP）
One often finds in the literature connections between measures of fairness and measures of feature importance employed to interpret trained classifiers. However, there seems to be no study that compares fairness measures and feature importance measures. In this paper we propose ways to evaluate and compare such measures. We focus in particular on SHAP, a game-theoretic measure of feature importance; we present results for a number of unfairness-prone datasets.
SoK: Taming the Triangle -- On the Interplays between Fairness, Interpretability and Privacy in Machine Learning
LAAS-CNRS, Université de Toulouse, CNRS（LAAS-CNRS，图卢兹大学，法国国家科学研究中心）；École de Technologie Supérieure（高等工程技术学院）；Université du Québec à Montréal（魁北克大学蒙特利尔分校）
Fused Gromov-Wasserstein Distance with Feature Selection
School of Data Science and Society（数据科学与社会学院）；Division of Computational Social Science（计算社会科学系）；University of North Carolina at Chapel Hill（北卡罗来纳大学教堂山分校）；Chinese University of Hong Kong, Shenzhen（香港中文大学（深圳））；Courant Institute（Courant研究所）；Department of Mathematics（数学系）；New York University（纽约大学）；Duke University（杜克大学）
Towards Cohesion-Fairness Harmony: Contrastive Regularization in Individual Fair Graph Clustering
L3S Research Centre, Leibniz University Hannover（莱布尼茨汉诺威大学L3S研究中心）；University of Kurdistan（库尔德斯坦大学）；RI CODE, University of the Bundeswehr Munich（慕尼黑联邦国防军大学RI CODE）
Xi’an Jiaotong-Liverpool University（西交利物浦大学）；Xi’an Jiaotong University（西安交通大学）；University of Oklahoma（俄克拉荷马大学）；University of Washington（华盛顿大学）
Scoring systems, as a type of predictive model, have significant advantages in interpretability and transparency and facilitate quick decision-making. As such, scoring systems have been extensively used in a wide variety of industries such as healthcare and criminal justice. However, the fairness issues in these models have long been criticized, and the use of big data and machine learning algorithms in the construction of scoring systems heightens this concern. In this paper, we propose a general framework to create fairness-aware, data-driven scoring systems. First, we develop a social welfa…
Evaluating Black-Box Vulnerabilities with Wasserstein-Constrained Data Perturbations
INRIA Regalia & ANITI Institut de Mathématiques de Toulouse, France Centre INRIA de l’Université de Bordeaux（INRIA里加大学与ANITI数学研究所，法国图卢兹数学研究所，INRIA波尔多中心）
Interpretable and Fair Generalized Additive Neural Networks via Multi-objective Learning
Southern University of Science and Technology（南方科技大学）；Beijing Normal-Hong Kong Baptist University（北京师范大学-香港浸会大学联合国际学院）；Agency for Science, Technology and Research (A*STAR)（新加坡科技研究局）；Nanyang Technological University（南洋理工大学）；Lingnan University（岭南大学）
Balancing Performance and Fairness in Explainable AI for Anomaly Detection in Distributed Power Plants Monitoring
African Institute for Mathematical Sciences (AIMS)（非洲数学科学研究所）；Rhodes University（罗德斯大学）；National Institute for Theoretical and Computational Sciences (NITheCS)（国家理论与计算科学研究所）；Ludwig Maximilian University of Munich（慕尼黑路易斯·马克西米利安大学）；University of Ferrara（费拉拉大学）
WGFormer: An SE(3)-Transformer Driven by Wasserstein Gradient Flows for Molecular Ground-State Conformation Prediction
Gaoling School of Artificial Intelligence, Renmin University of China（中国人民大学首都人工智能学院）；Beijing Key Laboratory of Research on Large Models（北京大型模型研究关键实验室）；Engineering Research Center of Next-Generation Intelligent Search（下一代智能搜索工程研究中心）
The interpretability of machine learning models has been an essential area of research for the safe deployment of machine learning systems. One particular approach is to attribute model decisions to high-level concepts that humans can understand. However, such concept-based explainability for Deep Neural Networks (DNNs) has been studied mostly on image domain. In this paper, we extend TCAV, the concept attribution approach, to tabular learning, by providing an idea on how to define concepts over tabular data. On a synthetic dataset with ground-truth concept explanations and a real-world datase…
Cornell University（康奈尔大学）；University of Chicago Booth School of Business（芝加哥大学布斯商学院）
Algorithms are increasingly used to aid, or in some cases supplant, human decision-making, particularly for decisions that hinge on predictions. As a result, two additional features in addition to prediction quality have generated interest: (i) to facilitate human interaction and understanding with these algorithms, we desire prediction functions that are in some fashion simple or interpretable; and (ii) because they influence consequential decisions, we also want them to produce equitable allocations. We develop a formal model to explore the relationship between the demands of simplicity and …
Adversarial Reweighting Guided by Wasserstein Distance for Bias Mitigation
SCHUFA Holding AG（舒法控股公司）；The Open University（开放大学）；L3S research center Hannover（汉诺威L3S研究中心）；University of Stuttgart（斯图加特大学）；Technical University of Munich（慕尼黑工业大学）
Google Research（谷歌研究院）；University of Cambridge（剑桥大学）；University of Copenhagen（哥本哈根大学）
The prototypical NLP experiment trains a standard architecture on labeled English data and optimizes for accuracy, without accounting for other dimensions such as fairness, interpretability, or computational efficiency. We show through a manual classification of recent NLP research papers that this is indeed the case and refer to it as the square one experimental setup. We observe that NLP research often goes beyond the square one setup, e.g, focusing not only on accuracy, but also on fairness or interpretability, but typically only along a single dimension. Most work targeting multilinguality…
AdapFair: Ensuring Adaptive Fairness for Machine Learning Operations
Department of Information Systems and Intelligent Business, School of Management, Xi’an Jiaotong University（信息系统与智能商业系，管理学院，西安交通大学）；School of Statistics and Data Science, Shanghai University of Finance and Economics（统计与数据科学学院，上海财经大学）
Wasserstein Wormhole: Scalable Optimal Transport Distance with Transformers
Sloan Kettering Institute（斯隆凯特琳研究所）；Memorial Sloan Kettering Cancer Center（纪念斯隆凯特琳癌症中心）；Tri-Institutional Training Program in Computational Biology and Medicine（三机构计算生物学与医学培训项目）；Weill Cornell Medicine（威尔康奈尔医学院）；Columbia University（哥伦比亚大学）；Yale University（耶鲁大学）；Howard Hughes Medical Institute（霍华德·休斯医学研究所）
The Maximum Mean Discrepancy (MMD) has found numerous applications in statistics and machine learning, most recently as a penalty in the Wasserstein Auto-Encoder (WAE). In this paper we compute closed-form expressions for estimating the Gaussian kernel based MMD between a given distribution and the standard multivariate normal distribution. This formula reveals a connection to the Baringhaus-Henze-Epps-Pulley (BHEP) statistic of the Henze-Zirkler test and provides further insights about the MMD. We introduce the standardized version of MMD as a penalty for the WAE training objective, allowing …
On the use of graph models to achieve individual and group fairness
Department of Statistics. Universidad Carlos III de Madrid（统计系. 皇家卡洛斯三世大学）；uc3m-Santander Big Data Institute. Universidad Carlos III de Madrid（uc3m-桑坦德大数据研究所. 皇家卡洛斯三世大学）
In recent years, there has been increasing interest in causal reasoning for designing fair decision-making systems due to its compatibility with legal frameworks, interpretability for human stakeholders, and robustness to spurious correlations inherent in observational data, among other factors. The recent attention to causal fairness, however, has been accompanied with great skepticism due to practical and epistemological challenges with applying current causal fairness approaches in the literature. Motivated by the long-standing empirical work on causality in econometrics, social sciences, a…
Algorithms are now routinely used to make consequential decisions that affect human lives. Examples include college admissions, medical interventions or law enforcement. While algorithms empower us to harness all information hidden in vast amounts of data, they may inadvertently amplify existing biases in the available datasets. This concern has sparked increasing interest in fair machine learning, which aims to quantify and mitigate algorithmic discrimination. Indeed, machine learning models should undergo intensive tests to detect algorithmic biases before being deployed at scale. In this pa…
Fair representation learning is an attractive approach that promises fairness of downstream predictors by encoding sensitive data. Unfortunately, recent work has shown that strong adversarial predictors can still exhibit unfairness by recovering sensitive attributes from these representations. In this work, we present Fair Normalizing Flows (FNF), a new approach offering more rigorous fairness guarantees for learned representations. Specifically, we consider a practical setting where we can estimate the probability density for sensitive groups. The key idea is to model the encoder as a normali…
Legislation and public sentiment throughout the world have promoted fairness metrics, explainability, and interpretability as prescriptions for the responsible development of ethical artificial intelligence systems. Despite the importance of these three pillars in the foundation of the field, they can be challenging to operationalize and attempts to solve the problems in production environments often feel Sisyphean. This difficulty stems from a number of factors: fairness metrics are computationally difficult to incorporate into training and rarely alleviate all of the harms perpetrated by the…
With the rise of AI, algorithms have become better at learning underlying patterns from the training data including ingrained social biases based on gender, race, etc. Deployment of such algorithms to domains such as hiring, healthcare, law enforcement, etc. has raised serious concerns about fairness, accountability, trust and interpretability in machine learning algorithms. To alleviate this problem, we propose D-BIAS, a visual interactive tool that embodies human-in-the-loop AI approach for auditing and mitigating social biases from tabular datasets. It uses a graphical causal model to repre…
