In this paper, we study the problem of fair classification in the presence of prior probability shifts, where the training set distribution differs from the test set. This phenomenon can be observed in the yearly records of several real-world datasets, such as recidivism records and medical expenditure surveys. If unaccounted for, such shifts can cause the predictions of a classifier to become unfair towards specific population subgroups. While the fairness notion called Proportional Equality (PE) accounts for such shifts, a procedure to ensure PE-fairness was unknown.
In this work, we propose a method, called CAPE, which provides a comprehensive solution to the aforementioned problem. CAPE makes novel use of prevalence estimation techniques, sampling and an ensemble of classifiers to ensure fair predictions under prior probability shifts. We introduce a metric, called prevalence difference (PD), which CAPE attempts to minimize in order to ensure PE-fairness. We theoretically establish that this metric exhibits several desirable properties.
We evaluate the efficacy of CAPE via a thorough empirical evaluation on synthetic datasets. We also compare the performance of CAPE with several popular fair classifiers on real-world datasets like COMPAS (criminal risk assessment) and MEPS (medical expenditure panel survey). The results indicate that CAPE ensures PE-fair predictions, while performing well on other performance metrics.
London School of Economics（伦敦经济学院）；Paris School of Economics（巴黎经济学院）
The paper introduces concepts of fairness and explainability (XAI) in artificial intelligence, oriented to solve a sophisticated business problems. For fairness, the authors discuss the bias-inducing specifics, as well as relevant mitigation methods, concluding with a set of recipes for introducing fairness in data-driven organizations. Additionally, for XAI, the authors audit specific algorithms paired with demonstrational business use-cases, discuss a plethora of techniques of explanations quality quantification and provide an overview of future research avenues.
AdapFair: Ensuring Adaptive Fairness for Machine Learning Operations
Department of Information Systems and Intelligent Business, School of Management, Xi’an Jiaotong University（信息系统与智能商业系，管理学院，西安交通大学）；School of Statistics and Data Science, Shanghai University of Finance and Economics（统计与数据科学学院，上海财经大学）
University of Maryland, College Park（马里兰大学帕克分校）；Boston University（波士顿大学）
The increasing reliance on ML models in high-stakes tasks has raised a major concern on fairness violations. Although there has been a surge of work that improves algorithmic fairness, most of them are under the assumption of an identical training and test distribution. In many real-world applications, however, such an assumption is often violated as previously trained fair models are often deployed in a different environment, and the fairness of such models has been observed to collapse. In this paper, we study how to transfer model fairness under distribution shifts, a widespread issue in pr…
University of Michigan（密歇根大学）；University of Konstanz（康斯坦茨大学）；IBM Research（IBM研究院）；Princeton University（普林斯顿大学）；Stanford University（斯坦福大学）
Many instances of algorithmic bias are caused by distributional shifts. For example, machine learning (ML) models often perform worse on demographic groups that are underrepresented in the training data. In this paper, we leverage this connection between algorithmic fairness and distribution shifts to show that algorithmic fairness interventions can help ML models overcome distribution shifts, and that domain adaptation methods (for overcoming distribution shifts) can mitigate algorithmic biases. In particular, we show that (i) enforcing suitable notions of individual fairness (IF) can improve…
Given an algorithmic predictor that is "fair" on some source distribution, will it still be fair on an unknown target distribution that differs from the source within some bound? In this paper, we study the transferability of statistical group fairness for machine learning predictors (i.e., classifiers or regressors) subject to bounded distribution shifts. Such shifts may be introduced by initial training data uncertainties, user adaptation to a deployed predictor, dynamic environments, or the use of pre-trained models in new settings. Herein, we develop a bound that characterizes such transfe…
L3S Research Center（L3S研究中心）；Leibniz Universität Hannover（汉诺威莱布尼茨大学）；Freie Universität Berlin（柏林自由大学）；The Open University（开放大学）
With the ever growing involvement of data-driven AI-based decision making technologies in our daily social lives, the fairness of these systems is becoming a crucial phenomenon. However, an important and often challenging aspect in utilizing such systems is to distinguish validity for the range of their application especially under distribution shifts, i.e., when a model is deployed on data with different distribution than the training set. In this paper, we present a case study on the newly released American Census datasets, a reconstruction of the popular Adult dataset, to illustrate the imp…
Supervised Algorithmic Fairness in Distribution Shifts: A Survey
School of New Media and Communication, Tianjin University（天津大学新媒体与传播学院）；College of Intelligence and Computing, Tianjin University（天津大学智能与计算学部）；Department of Computer Science, Baylor University（贝勒大学计算机系）；Electrical Engineering and Computer Science Department, University of Arkansas（阿肯色大学电气与计算机科学系）
University of Texas at Austin（德克萨斯大学奥斯汀分校）；Michigan State University（密歇根州立大学）
Increasing concerns have been raised on deep learning fairness in recent years. Existing fairness-aware machine learning methods mainly focus on the fairness of in-distribution data. However, in real-world applications, it is common to have distribution shift between the training and test data. In this paper, we first show that the fairness achieved by existing methods can be easily broken by slight distribution shifts. To solve this problem, we propose a novel fairness learning method termed CUrvature MAtching (CUMA), which can achieve robust fairness generalizable to unseen domains with unkn…
A review of clustering models in educational data science towards fairness-aware learning
L3S Research Center, Leibniz University Hannover（汉诺威莱布尼茨大学L3S研究中心）；Leibniz University Hannover（汉诺威莱布尼茨大学）；Institute for Didactics of Mathematics and Physics, Leibniz University Hannover（汉诺威莱布尼茨大学数学与物理教学研究所）；Research Institute CODE, Bundeswehr University Munich（慕尼黑联邦国防军大学CODE研究所）
Columbia University（哥伦比亚大学）；Department of Computer Science and Data Science Institute（计算机科学与数据科学研究所）
We initiate the study of fair classifiers that are robust to perturbations in the training distribution. Despite recent progress, the literature on fairness has largely ignored the design of fair and robust classifiers. In this work, we develop classifiers that are fair not only with respect to the training distribution, but also for a class of distributions that are weighted perturbations of the training samples. We formulate a min-max objective function whose goal is to minimize a distributionally robust training loss, and at the same time, find a classifier that is fair with respect to a cl…
Fix Representation (Optimally) Before Fairness: Finite-Sample Shrinkage Population Correction and the True Price of Fairness Under Subpopulation Shift
Department of Biostatistics, Vanderbilt University Medical Center, Nashville, TN 37232, USA（生物统计学系，范德比尔特大学医学中心）；Department of Informatics, King's College London, WC2R 2LS London, UK（信息学系，国王学院伦敦）
From Fragile to Certified: Wasserstein Audits of Group Fairness Under Distribution Shift
Max Planck Institute for Intelligent Systems（马克斯·普朗克智能系统研究所）；Tübingen AI Center（图宾根人工智能中心）；Mila Québec AI Institute（魁北克人工智能研究所）；McGill University（麦吉尔大学）
We study the problem of learning fair prediction models for unseen test sets distributed differently from the train set. Stability against changes in data distribution is an important mandate for responsible deployment of models. The domain adaptation literature addresses this concern, albeit with the notion of stability limited to that of prediction accuracy. We identify sufficient conditions under which stable models, both in terms of prediction accuracy and fairness, can be learned. Using the causal graph describing the data and the anticipated shifts, we specify an approach based on featur…
Combining the preferences of many rankers into one single consensus ranking is critical for consequential applications from hiring and admissions to lending. While group fairness has been extensively studied for classification, group fairness in rankings and in particular rank aggregation remains in its infancy. Recent work introduced the concept of fair rank aggregation for combining rankings but restricted to the case when candidates have a single binary protected attribute, i.e., they fall into two groups only. Yet it remains an open problem how to create a consensus ranking that represents…
University of Michigan（密歇根大学）；IBM Research（IBM研究院）；MIT-IBM Watson AI Lab（麻省理工学院-IBM沃森人工智能实验室）
Many instances of algorithmic bias are caused by subpopulation shifts. For example, ML models often perform worse on demographic groups that are underrepresented in the training data. In this paper, we study whether enforcing algorithmic fairness during training improves the performance of the trained model in the \emph{target domain}. On one hand, we conceive scenarios in which enforcing fairness does not improve performance in the target domain. In fact, it may even harm performance. On the other hand, we derive necessary and sufficient conditions under which enforcing algorithmic fairness l…
Existing algorithms for ensuring fairness in AI use a single-shot training strategy, where an AI model is trained on an annotated training dataset with sensitive attributes and then fielded for utilization. This training strategy is effective in problems with stationary distributions, where both training and testing data are drawn from the same distribution. However, it is vulnerable with respect to distributional shifts in the input space that may occur after the initial training phase. As a result, the time-dependent nature of data can introduce biases into the model predictions. Model retra…
Towards Collaborative Fairness in Federated Learning Under Imbalanced Covariate Shift
The Pennsylvania State University（宾夕法尼亚州立大学）；State University of New York at Albany（纽约州立大学阿尔巴尼分校）；University of Minnesota Twin Cities（明尼苏达大学双城分校）；Dalian University of Technology（大连理工大学）
KAIST（韩国科学技术院）；University of Wisconsin-Madison（威斯康星大学麦迪逊分校）
Model fairness is an essential element for Trustworthy AI. While many techniques for model fairness have been proposed, most of them assume that the training and deployment data distributions are identical, which is often not true in practice. In particular, when the bias between labels and sensitive groups changes, the fairness of the trained model is directly influenced and can worsen. We make two contributions for solving this problem. First, we analytically show that existing in-processing fair algorithms have fundamental limits in accuracy and group fairness. We introduce the notion of co…
FADE: Towards Fairness-aware Generation for Domain Generalization via Classifier-Guided Score-based Diffusion Models
School of New Media and Communication, Tianjin University（天津大学新媒体与传播学院）；School of Informatics, Xiamen University（厦门大学信息学院）；Department of Computer Science, Baylor University（贝勒大学计算机科学系）；Departments of Biostatistics and Epidemiology, Harvard University（哈佛大学生物统计学与流行病学系）
University of Illinois at Chicago（伊利诺伊大学芝加哥分校）；California Institute of Technology（加州理工学院）
Making predictions that are fair with regard to protected group membership (race, gender, age, etc.) has become an important requirement for classification algorithms. Existing techniques derive a fair model from sampled labeled data relying on the assumption that training and testing data are identically and independently drawn (iid) from the same distribution. In practice, distribution shift can and does occur between training and testing datasets as the characteristics of individuals interacting with the machine learning system change. We investigate fairness under covariate shift, a relaxa…
FairFedMed: Benchmarking Group Fairness in Federated Medical Imaging with FairLoRA
Harvard AI and Robotics Lab（哈佛人工智能与机器人实验室）；Harvard University（哈佛大学）；Harvard Ophthalmology AI lab（哈佛眼科人工智能实验室）；University of Central Florida（佛罗里达中央大学）；University of Louisiana at Lafayette（路易斯安那州立大学拉斐特分校）；New York University（纽约大学）；NYUAD Center for Artificial Intelligence and Robotics（纽约大学阿布扎克人工智能与机器人中心）
What is Fair? Defining Fairness in Machine Learning for Health
Department of Statistical Sciences, University of Toronto（统计科学系，多伦多大学）；Harvard T.H. Chan School of Public Health（哈佛T.H. Chan公共卫生学院）；York University（约克大学）；Verily Life Sciences（Verily生命科学公司）；Duke University（杜克大学）；University of Toronto（多伦多大学）
University of Texas at Austin（德克萨斯大学奥斯汀分校）；University of California, Berkeley（加利福尼亚大学伯克利分校）
We study the problem of best arm identification with a fairness constraint in a given causal model. The goal is to find a soft intervention on a given node to maximize the outcome while meeting a fairness constraint by counterfactual estimation with only partial knowledge of the causal model. The problem is motivated by ensuring fairness on an online marketplace. We provide theoretical guarantees on the probability of error and empirically examine the effectiveness of our algorithm with a two-stage baseline.
Qatar Computing Research Institute（卡塔尔计算研究所）；University of Gabes（加贝斯大学）；Sidra Medicine（西德拉医院）
Patient no-shows is a major burden for health centers leading to loss of revenue, increased waiting time and deteriorated health outcome. Developing machine learning (ML) models for the prediction of no -shows could help addressing this important issue. It is crucial to consider fair ML models for no-show prediction in order to ensure equality of opportunity in accessing healthcare services. In this wo rk, we are interested in developing deep learning models for no-show prediction based on tabular data while ensuring fairness properties. Our baseline model, TabNet, uses on attentive feature tr…
Many popular algorithmic fairness measures depend on the joint distribution of predictions, outcomes, and a sensitive feature like race or gender. These measures are sensitive to distribution shift: a predictor which is trained to satisfy one of these fairness definitions may become unfair if the distribution changes. In performative prediction settings, however, predictors are precisely intended to induce distribution shift. For example, in many applications in criminal justice, healthcare, and consumer finance, the purpose of building a predictor is to reduce the rate of adverse outcomes suc…
Utah State University（犹他州立大学）；University of Arkansas（阿肯色大学）；Clemson University（克莱姆森大学）
Ensuring fairness in anomaly detection models has received much attention recently as many anomaly detection applications involve human beings. However, existing fair anomaly detection approaches mainly focus on association-based fairness notions. In this work, we target counterfactual fairness, which is a prevalent causation-based fairness notion. The goal of counterfactually fair anomaly detection is to ensure that the detection outcome of an individual in the factual world is the same as that in the counterfactual world where the individual had belonged to a different group. To this end, we…
Istituto Italiano di Tecnologia（意大利理工学院）；University of Genova（热那亚大学）；University of Waterloo（滑铁卢大学）；University College London（伦敦大学学院）
We address the problem of algorithmic fairness: ensuring that sensitive variables do not unfairly influence the outcome of a classifier. We present an approach based on empirical risk minimization, which incorporates a fairness constraint into the learning problem. It encourages the conditional risk of the learned classifier to be approximately constant with respect to the sensitive variable. We derive both risk and fairness bounds that support the statistical consistency of our approach. We specify our approach to kernel methods and observe that the fairness requirement implies an orthogonali…
Face4FairShifts: A Large Image Benchmark for Fairness and Robust Learning across Visual Domains
School of New Media and Communication, Tianjin University（天津大学新媒体与传播学院）；Department of Computer Science, Baylor University（贝勒大学计算机科学系）；Electrical Engineering and Computer Science Department, University of Arkansas（阿肯色大学电子工程与计算机科学系）；NEC Laboratories America（NEC美国实验室）；School of Computing, Southern Illinois University（南伊利诺伊大学计算机学院）
