Our study revisits the problem of accuracy-fairness tradeoff in binary classification. We argue that comparison of non-discriminatory classifiers needs to account for different rates of positive predictions, otherwise conclusions about performance may be misleading, because accuracy and discrimination of naive baselines on the same dataset vary with different rates of positive predictions. We provide methodological recommendations for sound comparison of non-discriminatory classifiers, and present a brief theoretical and empirical analysis of tradeoffs between accuracy and non-discrimination.
School of Electrical and Computer Engineering, Purdue University（普渡大学电气与计算机工程学院）；Krannert School of Management, Purdue University（普渡大学克兰纳特管理学院）
The fairness in machine learning is getting increasing attention, as its applications in different fields continue to expand and diversify. To mitigate the discriminated model behaviors between different demographic groups, we introduce a novel post-processing method to optimize over multiple fairness constraints through group-aware threshold adaptation. We propose to learn adaptive classification thresholds for each demographic group by optimizing the confusion matrix estimated from the probability distribution of a classification model output. As we only need an estimated probability distrib…
Artificial neural nets can represent and classify many types of data but are often tailored to particular applications -- e.g., for "fair" or "hierarchical" classification. Once an architecture has been selected, it is often difficult for humans to adjust models for a new task; for example, a hierarchical classifier cannot be easily transformed into a fair classifier that shields a protected field. Our contribution in this work is a new neural network architecture, the concept subspace network (CSN), which generalizes existing specialized classifiers to produce a unified model capable of learn…
Beyond Independent Manipulation: Individual Fairness-aware Strategic Classification with Peer Imitation
College of Computer Science and Technology, National University of Defense Technology（国防科技大学计算机科学与技术学院）；School of Mathematical Sciences, Peking University（北京大学数学学院）；Institute for Theoretical Computer Science, Shanghai University of Finance and Economics（上海财经大学理论计算机科学研究所）；Information Technology Development, Aetos Capital Group, Sydney（悉尼Aetos资本集团信息技术部）；Faculty of Computing, Harbin Institute of Technology（哈尔滨工业大学计算机学院）；Department of Computer Science and Technology, Tsinghua University（清华大学计算机科学与技术系）
University of Southern California（南加州大学）；Information Sciences Institute（信息科学研究所）
Machine learning systems have been shown to propagate the societal errors of the past. In light of this, a wealth of research focuses on designing solutions that are "fair." Even with this abundance of work, there is no singular definition of fairness, mainly because fairness is subjective and context dependent. We propose a new fairness definition, motivated by the principle of equity, that considers existing biases in the data and attempts to make equitable decisions that account for these previous historical biases. We formalize our definition of fairness, and motivate it with its appropria…
As deep image classification applications, e.g., face recognition, become increasingly prevalent in our daily lives, their fairness issues raise more and more concern. It is thus crucial to comprehensively test the fairness of these applications before deployment. Existing fairness testing methods suffer from the following limitations: 1) applicability, i.e., they are only applicable for structured data or text without handling the high-dimensional and abstract domain sampling in the semantic level for image classification applications; 2) functionality, i.e., they generate unfair samples with…
Recent years have witnessed increasing concerns towards unfair decisions made by machine learning algorithms. To improve fairness in model decisions, various fairness notions have been proposed and many fairness-aware methods are developed. However, most of existing definitions and methods focus only on single-label classification. Fairness for multi-label classification, where each instance is associated with more than one labels, is still yet to establish. To fill this gap, we study fairness-aware multi-label classification in this paper. We start by extending Demographic Parity (DP) and Equ…
Fairness-aware classification is receiving increasing attention in the machine learning fields. Recently research proposes to formulate the fairness-aware classification as constrained optimization problems. However, several limitations exist in previous works due to the lack of a theoretical framework for guiding the formulation. In this paper, we propose a general framework for learning fair classifiers which addresses previous limitations. The framework formulates various commonly-used fairness metrics as convex constraints that can be directly incorporated into classic classification model…
The University of Melbourne（墨尔本大学）；Huawei Technologies Co., Ltd.（华为技术有限公司）；MBZUAI（穆罕默德·本·扎耶德人工智能大学）
This paper presents fairlib, an open-source framework for assessing and improving classification fairness. It provides a systematic framework for quickly reproducing existing baseline models, developing new methods, evaluating models with different metrics, and visualizing their results. Its modularity and extensibility enable the framework to be used for diverse types of inputs, including natural language, images, and audio. In detail, we implement 14 debiasing methods, including pre-processing, at-training-time, and post-processing approaches. The built-in metrics cover the most commonly use…
Data61 and the Australian National University（Data61 与澳大利亚国立大学）
We study the problem of learning classifiers with a fairness constraint, with three main contributions towards the goal of quantifying the problem's inherent tradeoffs. First, we relate two existing fairness measures to cost-sensitive risks. Second, we show that for cost-sensitive classification and fairness measures, the optimal classifier is an instance-dependent thresholding of the class-probability function. Third, we show how the tradeoff between accuracy and fairness is determined by the alignment between the class-probabilities for the target and sensitive features. Underpinning our ana…
Demystifying the Optimal Fair Classifier in Multi-Class Classification
College of Computer Science and Technology, Zhejiang University（浙江大学计算机科学与技术学院）；College of Computer Science（计算机科学学院）；Technology, Zhejiang University（技术，浙江大学）；School of Communication Engineering, Hangzhou Dianzi University（杭州电子科技大学通信工程学院）
A Generalised Exponentiated Gradient Approach to Enhance Fairness in Binary and Multi-class Classification Tasks
Laboratoire d’Intelligence Artificielle et Systèmes Industriels (LIASI), HESTIM Engineering and Business School（人工智能与工业系统实验室（LIASI），HESTIM工程与商业学院）；Laboratory LAMAI, Faculty of Sciences and Technologies, Cadi Ayyad University（LAMAI实验室，科学与技术学院，卡迪·阿亚德大学）；Department of Information Engineering, Computer Science and Mathematics, University of L'Aquila（信息工程、计算机科学与数学系，拉奎拉大学）
In this paper, we study counterfactual fairness in text classification, which asks the question: How would the prediction change if the sensitive attribute referenced in the example were different? Toxicity classifiers demonstrate a counterfactual fairness issue by predicting that "Some people are gay" is toxic while "Some people are straight" is nontoxic. We offer a metric, counterfactual token fairness (CTF), for measuring this particular form of fairness in text classifiers, and describe its relationship with group fairness. Further, we offer three approaches, blindness, counterfactual augm…
Do Fairness Interventions Come at the Cost of Privacy: Evaluations for Binary Classifiers
Australian Artificial Intelligence Institute and the School of Computer Science, University of Technology Sydney（澳大利亚人工智能研究所和悉尼技术大学计算机学院）；Data61, CSIRO（Data61和澳大利亚联邦科学与工业研究组织）；City University of Macau（澳门城市大学）
Educational technologies nowadays increasingly use data and Machine Learning (ML) models. This gives the students, instructors, and administrators support and insights for the optimum policy. However, it is well acknowledged that ML models are subject to bias, which raises concerns about the fairness, bias, and discrimination of using these automated ML algorithms in education and its unintended and unforeseen negative consequences. The contribution of bias during the decision-making comes from datasets used for training ML models and the model architecture. This paper presents a preliminary i…
QuantumBlack, a McKinsey company（QuantumBlack，麦肯锡旗下公司）；University of Warwick（华威大学）
Machine learning algorithms are extensively used to make increasingly more consequential decisions about people, so achieving optimal predictive performance can no longer be the only focus. A particularly important consideration is fairness with respect to race, gender, or any other sensitive attribute. This paper studies intersectional fairness, where intersections of multiple sensitive attributes are considered. Prior research has mainly focused on fairness with respect to a single sensitive attribute, with intersectional fairness being comparatively less studied despite its critical importa…
Developing classification algorithms that are fair with respect to sensitive attributes of the data has become an important problem due to the growing deployment of classification algorithms in various social contexts. Several recent works have focused on fairness with respect to a specific metric, modeled the corresponding fair classification problem as a constrained optimization problem, and developed tailored algorithms to solve them. Despite this, there still remain important metrics for which we do not have fair classifiers and many of the aforementioned algorithms do not come with theore…
KenSci Inc（KenSci公司）；University of Washington - Tacoma（华盛顿大学塔科马分校）；University of Washington - Bothell（华盛顿大学博塞尔分校）
Fairness in AI and machine learning systems has become a fundamental problem in the accountability of AI systems. While the need for accountability of AI models is near ubiquitous, healthcare in particular is a challenging field where accountability of such systems takes upon additional importance, as decisions in healthcare can have life altering consequences. In this paper we present preliminary results on fairness in the context of classification parity in healthcare. We also present some exploratory methods to improve fairness and choosing appropriate classification algorithms in the conte…
IBM Research（IBM研究院）；Massachusetts Institute of Technology（麻省理工学院）
Mitigating algorithmic bias is a critical task in the development and deployment of machine learning models. While several toolkits exist to aid machine learning practitioners in addressing fairness issues, little is known about the strategies practitioners employ to evaluate model fairness and what factors influence their assessment, particularly in the context of text classification. Two common approaches of evaluating the fairness of a model are group fairness and individual fairness. We run a study with Machine Learning practitioners (n=24) to understand the strategies used to evaluate mod…
University of California, Irvine（加州大学欧文分校）；Centers for Disease Control and Prevention（美国疾病控制与预防中心）
Applying standard machine learning approaches for classification can produce unequal results across different demographic groups. When then used in real-world settings, these inequities can have negative societal impacts. This has motivated the development of various approaches to fair classification with machine learning models in recent years. In this paper, we consider the problem of modifying the predictions of a blackbox machine learning classifier in order to achieve fairness in a multiclass setting. To accomplish this, we extend the 'post-processing' approach in Hardt et al. 2016, which…
TransFair: Transferring Fairness from Ocular Disease Classification to Progression Prediction
School of Computing and Informatics, University of Louisiana at Lafayette（路易斯安那大学拉斐特分校计算与信息学院）；Informatics Research Institute, University of Louisiana at Lafayette（路易斯安那大学拉斐特分校信息学研究所）；College of Engineering and Computer Science, Florida Atlantic University（佛罗里达大西洋大学工程与计算机科学学院）；Harvard Medical School（哈佛医学院）
Leibniz University, Hannover（汉诺威莱布尼茨大学）；L3S Research Center, Hannover（汉诺威L3S研究中心）
The wide spread usage of automated data-driven decision support systems has raised a lot of concerns regarding accountability and fairness of the employed models in the absence of human supervision. Existing fairness-aware approaches tackle fairness as a batch learning problem and aim at learning a fair model which can then be applied to future instances of the problem. In many applications, however, the data comes sequentially and its characteristics might evolve with time. In such a setting, it is counter-intuitive to "fix" a (fair) model over the data stream as changes in the data might inc…
Helmholtz AI（亥姆霍兹AI）；Ecole Polytechnique Fédérale de Lausanne（洛桑联邦理工学院）；Swisscom（瑞士电信）
The goal of fairness in classification is to learn a classifier that does not discriminate against groups of individuals based on sensitive attributes, such as race and gender. One approach to designing fair algorithms is to use relaxations of fairness notions as regularization terms or in a constrained optimization problem. We observe that the hyperbolic tangent function can approximate the indicator function. We leverage this property to define a differentiable relaxation that approximates fairness notions provably better than existing relaxations. In addition, we propose a model-agnostic mu…
Safe Fairness Guarantees Without Demographics in Classification: Spectral Uncertainty Set Perspective
Basque Center for Applied Mathematics (BCAM)（巴斯克应用数学中心）；Predictive Analytics Lab, University of Sussex（预测分析实验室，苏塞克斯大学）；Monash University Indonesia（墨尔本大学印尼分校）；University of the Basque Country EHU（巴斯克国家大学EHU）
The popularity of pretrained language models in natural language processing systems calls for a careful evaluation of such models in down-stream tasks, which have a higher potential for societal impact. The evaluation of such systems usually focuses on accuracy measures. Our findings in this paper call for attention to be paid to fairness measures as well. Through the analysis of more than a dozen pretrained language models of varying sizes on two toxic text classification tasks (English), we demonstrate that focusing on accuracy measures alone can lead to models with wide variation in fairnes…
Looking Beyond What You See: An Empirical Analysis on Subgroup Intersectional Fairness for Multi-label Chest X-ray Classification Using Social Determinants of Racial Health Inequities
Massachusetts Institute of Technology（麻省理工学院）；University at Buffalo（布法罗大学）
Robustness is of central importance in machine learning and has given rise to the fields of domain generalization and invariant learning, which are concerned with improving performance on a test distribution distinct from but related to the training distribution. In light of recent work suggesting an intimate connection between fairness and robustness, we investigate whether algorithms from robust ML can be used to improve the fairness of classifiers that are trained on biased data and tested on unbiased data. We apply Invariant Risk Minimization (IRM), a domain generalization algorithm that e…
University of California, Davis（加州大学戴维斯分校）；University of Virginia（弗吉尼亚大学）；University at Albany – State University of New York（纽约州立大学奥尔巴尼分校）
Existing work on fairness typically focuses on making known machine learning algorithms fairer. Fair variants of classification, clustering, outlier detection and other styles of algorithms exist. However, an understudied area is the topic of auditing an algorithm's output to determine fairness. Existing work has explored the two group classification problem for binary protected status variables using standard definitions of statistical parity. Here we build upon the area of auditing by exploring the multi-group setting under more complex definitions of fairness.
Uncovering Fairness through Data Complexity as an Early Indicator
Data Science and Computational Intelligence Institute (DaSCI), University of Granada（格拉纳达大学数据科学与计算智能研究所）；University of Bergen（卑尔根大学）；Department of Computer Science and Artificial Intelligence (DECSAI), University of Granada（格拉纳达大学计算机科学与人工智能系）
University of Illinois at Chicago（伊利诺伊大学芝加哥分校）；Carnegie Mellon University（卡内基梅隆大学）
Developing classification methods with high accuracy that also avoid unfair treatment of different groups has become increasingly important for data-driven decision making in social applications. Many existing methods enforce fairness constraints on a selected classifier (e.g., logistic regression) by directly forming constrained optimizations. We instead re-derive a new classifier from the first principles of distributional robustness that incorporates fairness criteria into a worst-case logarithmic loss minimization. This construction takes the form of a minimax game and produces a parametri…
Fairness has become increasingly pivotal in facial recognition. Without bias mitigation, deploying unfair AI would harm the interest of the underprivileged population. In this paper, we observe that though the higher accuracy that features from the deeper layer of a neural networks generally offer, fairness conditions deteriorate as we extract features from deeper layers. This phenomenon motivates us to extend the concept of multi-exit framework. Unlike existing works mainly focusing on accuracy, our multi-exit framework is fairness-oriented, where the internal classifiers are trained to be mo…
University of California, Los Angeles（加利福尼亚大学洛杉矶分校）
Existing pruning techniques preserve deep neural networks' overall ability to make correct predictions but may also amplify hidden biases during the compression process. We propose a novel pruning method, Fairness-aware GRAdient Pruning mEthod (FairGRAPE), that minimizes the disproportionate impacts of pruning on different sub-groups. Our method calculates the per-group importance of each model weight and selects a subset of weights that maintain the relative between-group total importance in pruning. The proposed method then prunes network edges with small importance values and repeats the pr…
UC Berkeley（加州大学伯克利分校）；Google Research Accra（谷歌研究院阿克拉分部）；Illinois（伊利诺伊大学）
In algorithmically fair prediction problems, a standard goal is to ensure the equality of fairness metrics across multiple overlapping groups simultaneously. We reconsider this standard fair classification problem using a probabilistic population analysis, which, in turn, reveals the Bayes-optimal classifier. Our approach unifies a variety of existing group-fair classification methods and enables extensions to a wide range of non-decomposable multiclass performance metrics and fairness measures. The Bayes-optimal classifier further inspires consistent procedures for algorithmically fair classi…
