Machine learning systems have been shown to propagate the societal errors of the past. In light of this, a wealth of research focuses on designing solutions that are "fair." Even with this abundance of work, there is no singular definition of fairness, mainly because fairness is subjective and context dependent. We propose a new fairness definition, motivated by the principle of equity, that considers existing biases in the data and attempts to make equitable decisions that account for these previous historical biases. We formalize our definition of fairness, and motivate it with its appropriate contexts. Next, we operationalize it for equitable classification. We perform multiple automatic and human evaluations to show the effectiveness of our definition and demonstrate its utility for aspects of fairness, such as the feedback loop.
Equity of educational outcome and fairness of AI with respect to race have been topics of increasing importance in education. In this work, we address both with empirical evaluations of grade prediction in higher education, an important task to improve curriculum design, plan interventions for academic support, and offer course guidance to students. With fairness as the aim, we trial several strategies for both label and instance balancing to attempt to minimize differences in algorithm performance with respect to race. We find that an adversarial learning approach, combined with grade label b…
Toyota Technological Institute at Chicago（芝加哥丰田技术研究所）；University of California at Irvine（加州大学欧文分校）
Machine Learning (ML) decision-making algorithms are now widely used in predictive decision-making, for example, to determine who to admit and give a loan. Their wide usage and consequential effects on individuals led the ML community to question and raise concerns on how the algorithms differently affect different people and communities. In this paper, we study fairness issues that arise when decision-makers use models (proxy models) that deviate from the models that depict the physical and social environment in which the decisions are situated (intended models). We also highlight the effect …
This study examines issues of algorithmic fairness in the context of systems that inform tax audit selection by the United States Internal Revenue Service (IRS). While the field of algorithmic fairness has developed primarily around notions of treating like individuals alike, we instead explore the concept of vertical equity -- appropriately accounting for relevant differences across individuals -- which is a central component of fairness in many public policy settings. Applied to the design of the U.S. individual income tax system, vertical equity relates to the fair allocation of tax and enf…
University of Vienna（维也纳大学）；Max-Planck Institute for Software Systems（马克斯·普朗克软件系统研究所）
We consider the equity and fairness of curricula derived from Knowledge Tracing models. We begin by defining a unifying notion of an equitable tutoring system as a system that achieves maximum possible knowledge in minimal time for each student interacting with it. Realizing perfect equity requires tutoring systems that can provide individualized curricula per student. In particular, we investigate the design of equitable tutoring systems that derive their curricula from Knowledge Tracing models. We first show that many existing models, including classical Bayesian Knowledge Tracing (BKT) and …
We study the interplay of fairness, welfare, and equity considerations in personalized pricing based on customer features. Sellers are increasingly able to conduct price personalization based on predictive modeling of demand conditional on covariates: setting customized interest rates, targeted discounts of consumer goods, and personalized subsidies of scarce resources with positive externalities like vaccines and bed nets. These different application areas may lead to different concerns around fairness, welfare, and equity on different objectives: price burdens on consumers, price envy, firm …
Exploring Bias and Prediction Metrics to Characterise the Fairness of Machine Learning for Equity-Centered Public Health Decision-Making: A Narrative Review
Vector Institute for Artificial Intelligence（人工智能向量研究院）；University of Tennessee（田纳西大学）；York University（约克大学）；McGill University（麦吉尔大学）
Cross-Institutional Transfer Learning for Educational Models: Implications for Model Performance, Fairness, and Equity
University of Washington（华盛顿大学）；Columbia University（哥伦比亚大学）；University of British Columbia（不列颠哥伦比亚大学）；University of Michigan（密歇根大学）；Cornell University（康奈尔大学）
Georgia Institute of Technology（佐治亚理工学院）；The Ohio State University（俄亥俄州立大学）
Recent research has demonstrated how racial biases against users who write African American English exists in popular toxic language datasets. While previous work has focused on a single fairness criteria, we propose to use additional descriptive fairness metrics to better understand the source of these biases. We demonstrate that different benchmark classifiers, as well as two in-process bias-remediation techniques, propagate racial biases even in a larger corpus. We then propose a novel ensemble-framework that uses a specialized classifier that is fine-tuned to the African American English d…
FairDiffusion: Enhancing Equity in Latent Diffusion Models via Fair Bayesian Perturbation
Harvard Medical School（哈佛医学院）；New York University（纽约大学）；New York University Abu Dhabi（纽约大学阿布扎比分校）；Harvard University（哈佛大学）；Broad Institute of MIT and Harvard（麻省理工学院与哈佛大学博德研究所）
We study fairness in collaborative-filtering recommender systems, which are sensitive to discrimination that exists in historical data. Biased data can lead collaborative-filtering methods to make unfair predictions for users from minority groups. We identify the insufficiency of existing fairness metrics and propose four new metrics that address different forms of unfairness. These fairness metrics can be optimized by adding fairness terms to the learning objective. Experiments on synthetic and real data show that our new metrics can better measure fairness than the baseline, and that the fai…
Helmholtz AI（亥姆霍兹AI）；Ecole Polytechnique Fédérale de Lausanne（洛桑联邦理工学院）；Swisscom（瑞士电信）
The goal of fairness in classification is to learn a classifier that does not discriminate against groups of individuals based on sensitive attributes, such as race and gender. One approach to designing fair algorithms is to use relaxations of fairness notions as regularization terms or in a constrained optimization problem. We observe that the hyperbolic tangent function can approximate the indicator function. We leverage this property to define a differentiable relaxation that approximates fairness notions provably better than existing relaxations. In addition, we propose a model-agnostic mu…
Federated Learning (FL) enables data owners to train a shared global model without sharing their private data. Unfortunately, FL is susceptible to an intrinsic fairness issue: due to heterogeneity in clients' data distributions, the final trained model can give disproportionate advantages across the participating clients. In this work, we present Equal and Equitable Federated Learning (E2FL) to produce fair federated learning models by preserving two main fairness properties, equity and equality, concurrently. We validate the efficiency and fairness of E2FL in different real-world FL applicati…
Carnegie Mellon University（卡内基梅隆大学）；Ctr for Data Science & Public Policy , University of Chicago（芝加哥大学数据科学与公共政策中心）；Los Angeles City Attorney’s Office（洛杉矶市检察官办公室）；Heinz College of Information Systems & Public Policy , Carnegie Mellon University（卡内基梅隆大学海因茨信息系统与公共政策学院）；Machine Learning Department , Carnegie Mellon University（卡内基梅隆大学机器学习系）；Director, Recidivism Reduction and Drug Diversion Unit (R2D2) , Los Angeles City Attorney’s Office（洛杉矶市检察官办公室再犯率降低和药物分流部门主任（R2D2））
The criminal justice system is currently ill-equipped to improve outcomes of individuals who cycle in and out of the system with a series of misdemeanor offenses. Often due to constraints of caseload and poor record linkage, prior interactions with an individual may not be considered when an individual comes back into the system, let alone in a proactive manner through the application of diversion programs. The Los Angeles City Attorney's Office recently created a new Recidivism Reduction and Drug Diversion unit (R2D2) tasked with reducing recidivism in this population. Here we describe a coll…
East China University of Science and Technology（华东理工大学）；Northeastern University（东北大学）；Bielefeld University（比勒费尔德大学）；University of Surrey（萨里大学）
Recent years have seen the rapid development of fairness-aware machine learning in mitigating unfairness or discrimination in decision-making in a wide range of applications. However, much less attention has been paid to the fairness-aware multi-objective optimization, which is indeed commonly seen in real life, such as fair resource allocation problems and data driven multi-objective optimization problems. This paper aims to illuminate and broaden our understanding of multi-objective optimization from the perspective of fairness. To this end, we start with a discussion of user preferences in …
Lehigh University（理海大学）；University of Coimbra（科英布拉大学）
In the application of machine learning to real-life decision-making systems, e.g., credit scoring and criminal justice, the prediction outcomes might discriminate against people with sensitive attributes, leading to unfairness. The commonly used strategy in fair machine learning is to include fairness as a constraint or a penalization term in the minimization of the prediction loss, which ultimately limits the information given to decision-makers. In this paper, we introduce a new approach to handle fairness by formulating a stochastic multi-objective optimization problem for which the corresp…
In this work we formulate and formally characterize group fairness as a multi-objective optimization problem, where each sensitive group risk is a separate objective. We propose a fairness criterion where a classifier achieves minimax risk and is Pareto-efficient w.r.t. all groups, avoiding unnecessary harm, and can lead to the best zero-gap model if policy dictates so. We provide a simple optimization algorithm compatible with deep neural networks to satisfy these constraints. Since our method does not require test-time access to sensitive attributes, it can be applied to reduce worst-case cl…
Equitable Survival Prediction: A Fairness-Aware Survival Modeling (FASM) Approach
Duke-NUS Medical School（杜克-新加坡国立大学医学院）；Duke University School of Medicine（杜克大学医学院）；Duke AI Health（杜克人工智能健康中心）；Mass General Brigham（麻省总医院布里格姆）；Harvard Medical School（哈佛医学院）；National University of Singapore（新加坡国立大学）
Emerging transportation modes, including car-sharing, bike-sharing, and ride-hailing, are transforming urban mobility but have been shown to reinforce socioeconomic inequities. Spatiotemporal demand prediction models for these new mobility regimes must therefore consider fairness as a first-class design requirement. We present FairST, a fairness-aware model for predicting demand for new mobility systems. Our approach utilizes 1D, 2D and 3D convolutions to integrate various urban features and learn the spatial-temporal dynamics of a mobility system, but we include fairness metrics as a form of …
Algorithmic fairness involves expressing notions such as equity, or reasonable treatment, as quantifiable measures that a machine learning algorithm can optimise. Most work in the literature to date has focused on classification problems where the prediction is categorical, such as accepting or rejecting a loan application. This is in part because classification fairness measures are easily computed by comparing the rates of outcomes, leading to behaviours such as ensuring that the same fraction of eligible men are selected as eligible women. But such measures are computationally difficult to …
University of Toronto（多伦多大学）；Vector Institute（向量研究所）
Deploying machine learning (ML) models often requires both fairness and privacy guarantees. Both of these objectives present unique trade-offs with the utility (e.g., accuracy) of the model. However, the mutual interactions between fairness, privacy, and utility are less well-understood. As a result, often only one objective is optimized, while the others are tuned as hyper-parameters. Because they implicitly prioritize certain objectives, such designs bias the model in pernicious, undetectable ways. To address this, we adopt impartiality as a principle: design of ML pipelines should not favor…
University of Milano-Bicocca（米兰比可卡大学）；OAKS srl（OAKS有限责任公司）
There is a consensus that focusing only on accuracy in searching for optimal machine learning models amplifies biases contained in the data, leading to unfair predictions and decision supports. Recently, multi-objective hyperparameter optimization has been proposed to search for machine learning models which offer equally Pareto-efficient trade-offs between accuracy and fairness. Although these approaches proved to be more versatile than fairness-aware machine learning algorithms -- which optimize accuracy constrained to some threshold on fairness -- they could drastically increase the energy …
Mitigating Unfairness via Evolutionary Multi-objective Ensemble Learning
Research Institute of Trustworthy Autonomous System, Southern University of Science and Technology（南方科技大学可信自主系统研究院）；Trustworthiness Theory Research Center, Huawei Technologies Co., Ltd.（华为技术有限公司可信理论研究中心）；School of Computer Science, University of Birmingham（伯明翰大学计算机学院）
Holistic Survey of Privacy and Fairness in Machine Learning
University of Southern California（南加州大学）；Deakin University（迪肯大学）；University of Alabama in Huntsville（阿拉巴马大学亨茨维尔分校）；Amazon（亚马逊）；Hamad Bin Khalifa University（哈马德·本·哈利法大学）
