We propose a model-agnostic approach for mitigating the prediction bias of a black-box decision-maker, and in particular, a human decision-maker. Our method detects in the feature space where the black-box decision-maker is biased and replaces it with a few short decision rules, acting as a "fair surrogate". The rule-based surrogate model is trained under two objectives, predictive performance and fairness. Our model focuses on a setting that is common in practice but distinct from other literature on fairness. We only have black-box access to the model, and only a limited set of true labels can be queried under a budget constraint. We formulate a multi-objective optimization for building a surrogate model, where we simultaneously optimize for both predictive performance and bias. To train the model, we propose a novel training algorithm that combines a nondominated sorting genetic algorithm with active learning. We test our model on public datasets where we simulate various biased "black-box" classifiers (decision-makers) and apply our approach for interpretable augmented fairness.
University of Tsukuba（筑波大学）；RIKEN AIP（理化学研究所人工智能研究中心）；Osaka University（大阪大学）
Auditing fairness of decision-makers is now in high demand. To respond to this social demand, several fairness auditing tools have been developed. The focus of this study is to raise an awareness of the risk of malicious decision-makers who fake fairness by abusing the auditing tools and thereby deceiving the social communities. The question is whether such a fraud of the decision-maker is detectable so that the society can avoid the risk of fake fairness. In this study, we answer this question negatively. We specifically put our focus on a situation where the decision-maker publishes a benchm…
FairContrast: Enhancing Fairness through Contrastive learning and Customized Augmenting Methods on Tabular Data
Department of Industrial Engineering University of Central Florida（工业工程系，中央佛罗里达大学）；Department of Computer Science University of Central Florida（计算机科学系，中央佛罗里达大学）
On Prediction-Modelers and Decision-Makers: Why Fairness Requires More Than a Fair Prediction Model
Ca’ Foscari University of Venice（威尼斯大学）；European Centre for Living Technology（欧洲生活技术中心）；University of Zurich（苏黎世大学）；Zurich University of Applied Sciences（苏黎世应用科技大学）
University of Toronto（多伦多大学）；Vector Institute（向量研究所）
In many machine learning applications, there are multiple decision-makers involved, both automated and human. The interaction between these agents often goes unaddressed in algorithmic development. In this work, we explore a simple version of this interaction with a two-stage framework containing an automated model and an external decision-maker. The model can choose to say "Pass", and pass the decision downstream, as explored in rejection learning. We extend this concept by proposing "learning to defer", which generalizes rejection learning by considering the effect of other agents in the dec…
University of Delaware（特拉华大学）；Ohio State University（俄亥俄州立大学）；Alan Turing Institute（艾伦·图灵研究所）
We consider a selection problem where sequentially arrived applicants apply for a limited number of positions/jobs. At each time step, a decision maker accepts or rejects the given applicant using a pre-trained supervised learning model until all the vacant positions are filled. In this paper, we discuss whether the fairness notions (e.g., equal opportunity, statistical parity, etc.) that are commonly used in classification problems are suitable for the sequential selection problems. In particular, we show that even with a pre-trained model that satisfies the common fairness notions, the selec…
Rethinking Algorithmic Fairness for Human-AI Collaboration
Wharton AI & Analytics Initiative, Wharton School, University of Pennsylvania（沃顿商学院人工智能与分析倡议，沃顿商学院，宾夕法尼亚大学）；Department of Operations, Information, and Decisions, Wharton School, University of Pennsylvania（运营、信息与决策系，沃顿商学院，宾夕法尼亚大学）；Department of Computer and Information Science, University of Pennsylvania（计算机与信息科学系，宾夕法尼亚大学）
Data-driven predictive models are increasingly used in education to support students, instructors, and administrators. However, there are concerns about the fairness of the predictions and uses of these algorithmic systems. In this introduction to algorithmic fairness in education, we draw parallels to prior literature on educational access, bias, and discrimination, and we examine core components of algorithmic systems (measurement, model learning, and action) to identify sources of bias and discrimination in the process of developing and deploying these systems. Statistical, similarity-based…
Univ. Grenoble Alpes（格勒诺布尔阿尔卑斯大学）；Inria（法国国家信息与自动化研究所）；CNRS（法国国家科学研究中心）；Grenoble INP（格勒诺布尔理工学院）；LIG（格勒诺布尔信息实验室）；Max Planck Institute for Software Systems（马克斯·普朗克软件系统研究所）
Discrimination in selection problems such as hiring or college admission is often explained by implicit bias from the decision maker against disadvantaged demographic groups. In this paper, we consider a model where the decision maker receives a noisy estimate of each candidate's quality, whose variance depends on the candidate's group -- we argue that such differential variance is a key feature of many selection problems. We analyze two notable settings: in the first, the noise variances are unknown to the decision maker who simply picks the candidates with the highest estimated quality indep…
Toyota Technological Institute at Chicago（芝加哥丰田技术研究所）；University of California at Irvine（加州大学欧文分校）
Machine Learning (ML) decision-making algorithms are now widely used in predictive decision-making, for example, to determine who to admit and give a loan. Their wide usage and consequential effects on individuals led the ML community to question and raise concerns on how the algorithms differently affect different people and communities. In this paper, we study fairness issues that arise when decision-makers use models (proxy models) that deviate from the models that depict the physical and social environment in which the decisions are situated (intended models). We also highlight the effect …
Lehigh University（理海大学）；University of Coimbra（科英布拉大学）
In the application of machine learning to real-life decision-making systems, e.g., credit scoring and criminal justice, the prediction outcomes might discriminate against people with sensitive attributes, leading to unfairness. The commonly used strategy in fair machine learning is to include fairness as a constraint or a penalization term in the minimization of the prediction loss, which ultimately limits the information given to decision-makers. In this paper, we introduce a new approach to handle fairness by formulating a stochastic multi-objective optimization problem for which the corresp…
Decision makers increasingly rely on algorithmic risk scores to determine access to binary treatments including bail, loans, and medical interventions. In these settings, we reconcile two fairness criteria that were previously shown to be in conflict: calibration and error rate equality. In particular, we derive necessary and sufficient conditions for the existence of calibrated scores that yield classifications achieving equal error rates at any given group-blind threshold. We then present an algorithm that searches for the most accurate score subject to both calibration and minimal error rat…
Yale University（耶鲁大学）；University of Texas at Austin（德克萨斯大学奥斯汀分校）；Amazon AWS AI（亚马逊云服务人工智能）；Fiddler AI
In hybrid human-machine deferral frameworks, a classifier can defer uncertain cases to human decision-makers (who are often themselves fallible). Prior work on simultaneous training of such classifier and deferral models has typically assumed access to an oracle during training to obtain true class labels for training samples, but in practice there often is no such oracle. In contrast, we consider a "closed" decision-making pipeline in which the same fallible human decision-makers used in deferral also provide training labels. How can imperfect and biased human expert labels be used to train a…
AT&T Chief Data Office（AT&T首席数据办公室）；Trustworthy ML Initiative（可信机器学习倡议）
New technologies and the availability of geospatial data have drawn attention to spatio-temporal biases present in society. For example: the COVID-19 pandemic highlighted disparities in the availability of broadband service and its role in the digital divide; the environmental justice movement in the United States has raised awareness to health implications for minority populations stemming from historical redlining practices; and studies have found varying quality and coverage in the collection and sharing of open-source geospatial data. Despite the extensive literature on machine learning (M…
University of Maryland（马里兰大学）；University of St. Thomas（圣托马斯大学）；Montgomery Blair High School（蒙哥马利布莱尔高中）；Microsoft Research（微软研究院）
We propose new tools for policy-makers to use when assessing and correcting fairness and bias in AI algorithms. The three tools are: - A new definition of fairness called "controlled fairness" with respect to choices of protected features and filters. The definition provides a simple test of fairness of an algorithm with respect to a dataset. This notion of fairness is suitable in cases where fairness is prioritized over accuracy, such as in cases where there is no "ground truth" data, only data labeled with past decisions (which may have been biased). - Algorithms for retraining a given class…
Toyota Technological Institute at Chicago（芝加哥丰田技术研究所）
Consider an actor making selection decisions using a series of classifiers, which we term a sequential screening process. The early stages filter out some applicants, and in the final stage an expensive but accurate test is applied to the individuals that make it to the final stage. Since the final stage is expensive, if there are multiple groups with different fractions of positives at the penultimate stage (even if a slight gap), then the firm may naturally only choose to the apply the final (interview) stage solely to the highest precision group which would be clearly unfair to the other gr…
University of Maryland（马里兰大学）；Tulane University（杜兰大学）
We propose a novel formulation of group fairness with biased feedback in the contextual multi-armed bandit (CMAB) setting. In the CMAB setting, a sequential decision maker must, at each time step, choose an arm to pull from a finite set of arms after observing some context for each of the potential arm pulls. In our model, arms are partitioned into two or more sensitive groups based on some protected feature(s) (e.g., age, race, or socio-economic status). Initial rewards received from pulling an arm may be distorted due to some unknown societal or measurement bias. We assume that in reality th…
A review of clustering models in educational data science towards fairness-aware learning
L3S Research Center, Leibniz University Hannover（汉诺威莱布尼茨大学L3S研究中心）；Leibniz University Hannover（汉诺威莱布尼茨大学）；Institute for Didactics of Mathematics and Physics, Leibniz University Hannover（汉诺威莱布尼茨大学数学与物理教学研究所）；Research Institute CODE, Bundeswehr University Munich（慕尼黑联邦国防军大学CODE研究所）
Society increasingly relies on machine learning models for automated decision making. Yet, efficiency gains from automation have come paired with concern for algorithmic discrimination that can systematize inequality. Recent work has proposed optimal post-processing methods that randomize classification decisions for a fraction of individuals, in order to achieve fairness measures related to parity in errors and calibration. These methods, however, have raised concern due to the information inefficiency, intra-group unfairness, and Pareto sub-optimality they entail. The present work proposes a…
University of Canterbury（坎特伯雷大学）；Corvinus University of Budapest（布达佩斯考维努斯大学）；Soul Machines（Soul Machines公司）；Universitat Pompeu Fabra（庞培法布拉大学）
As \emph{artificial intelligence} (AI) systems are increasingly involved in decisions affecting our lives, ensuring that automated decision-making is fair and ethical has become a top priority. Intuitively, we feel that akin to human decisions, judgments of artificial agents should necessarily be grounded in some moral principles. Yet a decision-maker (whether human or artificial) can only make truly ethical (based on any ethical theory) and fair (according to any notion of fairness) decisions if full information on all the relevant factors on which the decision is based are available at the t…
Valencian Research Institute for Artificial Intelligence (VRAIN)（瓦伦西亚人工智能研究所）；Universitat Politècnica de València（瓦伦西亚理工大学）
The causes underlying unfair decision making are complex, being internalised in different ways by decision makers, other actors dealing with data and models, and ultimately by the individuals being affected by these decisions. One frequent manifestation of all these latent causes arises in the form of missing values: protected groups are more reluctant to give information that could be used against them, delicate information for some groups can be erased by human operators, or data acquisition may simply be less complete and systematic for minority groups. As a result, missing values and bias …
University of Southern California（南加州大学）；Information Sciences Institute（信息科学研究所）；Amazon Alexa（亚马逊Alexa）；Harvard University（哈佛大学）；University of California, Los Angeles（加州大学洛杉矶分校）
Language models excel at generating coherent text, and model compression techniques such as knowledge distillation have enabled their use in resource-constrained settings. However, these models can be biased in multiple ways, including the unfounded association of male and female genders with gender-neutral professions. Therefore, knowledge distillation without any fairness constraints may preserve or exaggerate the teacher model's biases onto the distilled model. To this end, we present a novel approach to mitigate gender disparity in text generation by learning a fair model during knowledge …
Inspired by recent breakthroughs in predictive modeling, practitioners in both industry and government have turned to machine learning with hopes of operationalizing predictions to drive automated decisions. Unfortunately, many social desiderata concerning consequential decisions, such as justice or fairness, have no natural formulation within a purely predictive framework. In efforts to mitigate these problems, researchers have proposed a variety of metrics for quantifying deviations from various statistical parities that we might expect to observe in a fair world and offered a variety of alg…
FAIRPLAI: A Human-in-the-Loop Approach to Fair and Private Machine Learning
Dept. of Computer Science, The University of Texas at El Paso（得克萨斯大学埃尔帕索分校计算机科学系）；Dept. of Mathematics, Mountain View High School（山景高中数学系）；Dept. of Mathematics, El Paso Independent School District（埃尔帕索独立学区数学系）
This thesis scrutinizes common assumptions underlying traditional machine learning approaches to fairness in consequential decision making. After challenging the validity of these assumptions in real-world applications, we propose ways to move forward when they are violated. First, we show that group fairness criteria purely based on statistical properties of observed data are fundamentally limited. Revisiting this limitation from a causal viewpoint we develop a more versatile conceptual framework, causal fairness criteria, and first algorithms to achieve them. We also provide tools to analyze…
The influence of human judgement is ubiquitous in datasets used across the analytics industry, yet humans are known to be sub-optimal decision makers prone to various biases. Analysing biased datasets then leads to biased outcomes of the analysis. Bias by protected characteristics (e.g. race) is of particular interest as it may not only make the output of analytical process sub-optimal, but also illegal. Countering the bias by constraining the analytical outcomes to be fair is problematic because A) fairness lacks a universally accepted definition, while at the same time some definitions are m…
