Machine learning algorithms are extensively used to make increasingly more consequential decisions about people, so achieving optimal predictive performance can no longer be the only focus. A particularly important consideration is fairness with respect to race, gender, or any other sensitive attribute. This paper studies intersectional fairness, where intersections of multiple sensitive attributes are considered. Prior research has mainly focused on fairness with respect to a single sensitive attribute, with intersectional fairness being comparatively less studied despite its critical importance for the safety of modern machine learning systems. We present a comprehensive framework for auditing and achieving intersectional fairness in classification problems: we define a suite of metrics to assess intersectional fairness in the data or model outputs by extending known single-attribute fairness metrics, and propose methods for robustly estimating them even when some intersectional subgroups are underrepresented. Furthermore, we develop post-processing techniques to mitigate any detected intersectional bias in a classification model. Our techniques do not rely on any assumptions regarding the underlying model and preserve predictive performance at a guaranteed level of fairness. Finally, we give guidance on a practical implementation, showing how the proposed methods perform on a real-world dataset.
University of Illinois Urbana-Champaign（伊利诺伊大学厄巴纳-香槟分校）；Arizona State University（亚利桑那州立大学）；University of Arkansas（阿肯色大学）
Algorithmic fairness is becoming increasingly important in data mining and machine learning. Among others, a foundational notation is group fairness. The vast majority of the existing works on group fairness, with a few exceptions, primarily focus on debiasing with respect to a single sensitive attribute, despite the fact that the co-existence of multiple sensitive attributes (e.g., gender, race, marital status, etc.) in the real-world is commonplace. As such, methods that can ensure a fair learning outcome with respect to all sensitive attributes of concern simultaneously need to be developed…
Research in machine learning fairness has historically considered a single binary demographic attribute; however, the reality is of course far more complicated. In this work, we grapple with questions that arise along three stages of the machine learning pipeline when incorporating intersectionality as multiple demographic attributes: (1) which demographic attributes to include as dataset labels, (2) how to handle the progressively smaller size of subgroups during model training, and (3) how to move beyond existing evaluation metrics when benchmarking model fairness for more subgroups. For eac…
University of Tsukuba（筑波大学）；RIKEN AIP（理化学研究所人工智能研究中心）；Osaka University（大阪大学）
Auditing fairness of decision-makers is now in high demand. To respond to this social demand, several fairness auditing tools have been developed. The focus of this study is to raise an awareness of the risk of malicious decision-makers who fake fairness by abusing the auditing tools and thereby deceiving the social communities. The question is whether such a fraud of the decision-maker is detectable so that the society can avoid the risk of fake fairness. In this study, we answer this question negatively. We specifically put our focus on a situation where the decision-maker publishes a benchm…
University of Maryland, Baltimore County（马里兰大学巴尔的摩县分校）
Intersectionality is a framework that analyzes how interlocking systems of power and oppression affect individuals along overlapping dimensions including race, gender, sexual orientation, class, and disability. Intersectionality theory therefore implies it is important that fairness in artificial intelligence systems be protected with regard to multi-dimensional protected attributes. However, the measurement of fairness becomes statistically challenging in the multi-dimensional setting due to data sparsity, which increases rapidly in the number of dimensions, and in the values per dimension. W…
The growing capability and accessibility of machine learning has led to its application to many real-world domains and data about people. Despite the benefits algorithmic systems may bring, models can reflect, inject, or exacerbate implicit and explicit societal biases into their outputs, disadvantaging certain demographic subgroups. Discovering which biases a machine learning model has introduced is a great challenge, due to the numerous definitions of fairness and the large number of potentially impacted subgroups. We present FairVis, a mixed-initiative visual analytics system that integrate…
University of Pennsylvania（宾夕法尼亚大学）；Microsoft Research-New York City（微软研究院（纽约市））
The most prevalent notions of fairness in machine learning are statistical definitions: they fix a small collection of pre-defined groups, and then ask for parity of some statistic of the classifier across these groups. Constraints of this form are susceptible to intentional or inadvertent "fairness gerrymandering", in which a classifier appears to be fair on each individual group, but badly violates the fairness constraint on one or more structured subgroups defined over the protected attributes. We propose instead to demand statistical notions of fairness across exponentially (or infinitely)…
University of Maryland, Baltimore County（马里兰大学巴尔的摩县分校）
We propose definitions of fairness in machine learning and artificial intelligence systems that are informed by the framework of intersectionality, a critical lens arising from the Humanities literature which analyzes how interlocking systems of power and oppression affect individuals along overlapping dimensions including gender, race, sexual orientation, class, and disability. We show that our criteria behave sensibly for any subset of the set of protected attributes, and we prove economic, privacy, and generalization guarantees. We provide a learning algorithm which respects our intersectio…
Machine Learning or Artificial Intelligence algorithms have gained considerable scrutiny in recent times owing to their propensity towards imitating and amplifying existing prejudices in society. This has led to a niche but growing body of work that identifies and attempts to fix these biases. A first step towards making these algorithms more fair is designing metrics that measure unfairness. Most existing work in this field deals with either a binary view of fairness (protected vs. unprotected groups) or politically defined categories (race or gender). Such categorization misses the important…
Exposing the Illusion of Fairness: Auditing Vulnerabilities to Distributional Manipulation Attacks
Institut de Mathématiques de Toulouse (IMT)（图卢兹数学研究所）；Institut national de recherche en sciences et technologies du numérique (INRIA)（国家数字科学与技术研究院）；Artificial and Natural Intelligence Toulouse Institute 2 (ANITI 2)（图卢兹人工智能与自然智能研究所2）；Institut de Recherche en Informatique de Toulouse (IRIT)（图卢兹计算机研究所）；Independent Researcher（独立研究员）；CNRS IRL CROSSING（国家科学研究中心IRL CROSSING）
The issue of fairness in AI has received an increasing amount of attention in recent years. The problem can be approached by looking at different protected attributes (e.g., ethnicity, gender, etc) independently, but fairness for individual protected attributes does not imply intersectional fairness. In this work, we frame the problem of intersectional fairness within a geometrical setting. We project our data onto a hypercube, and split the analysis of fairness by levels, where each level encodes the number of protected attributes we are intersecting over. We prove mathematically that, while …
The ImageNet dataset ushered in a flood of academic and industry interest in deep learning for computer vision applications. Despite its significant impact, there has not been a comprehensive investigation into the demographic attributes of images contained within the dataset. Such a study could lead to new insights on inherent biases within ImageNet, particularly important given it is frequently used to pretrain models for a wide variety of computer vision tasks. In this work, we introduce a model-driven framework for the automatic annotation of apparent age and gender attributes in large-sca…
Looking Beyond What You See: An Empirical Analysis on Subgroup Intersectional Fairness for Multi-label Chest X-ray Classification Using Social Determinants of Racial Health Inequities
Massachusetts Institute of Technology（麻省理工学院）；University at Buffalo（布法罗大学）
Intersectional Divergence: Measuring Fairness in Regression
Lucy Family Institute for Data \& Society, University of Notre Dame Notre Dame IN USA；Lucy Family Institute for Data \& Society, University of Notre Dame
Combining the preferences of many rankers into one single consensus ranking is critical for consequential applications from hiring and admissions to lending. While group fairness has been extensively studied for classification, group fairness in rankings and in particular rank aggregation remains in its infancy. Recent work introduced the concept of fair rank aggregation for combining rankings but restricted to the case when candidates have a single binary protected attribute, i.e., they fall into two groups only. Yet it remains an open problem how to create a consensus ranking that represents…
Diversity Drives Fairness: Ensemble of Higher Order Mutants for Intersectional Fairness of Machine Learning Software
Nanyang Technological University（南洋理工大学）；Peking University（北京大学）；King’s College London（伦敦国王学院）；University College London（伦敦大学学院）；China-Singapore International Joint Research Institute（中新国际联合研究院）
University of California, Davis（加州大学戴维斯分校）；University of Virginia（弗吉尼亚大学）；University at Albany – State University of New York（纽约州立大学奥尔巴尼分校）
Existing work on fairness typically focuses on making known machine learning algorithms fairer. Fair variants of classification, clustering, outlier detection and other styles of algorithms exist. However, an understudied area is the topic of auditing an algorithm's output to determine fairness. Existing work has explored the two group classification problem for binary protected status variables using standard definitions of statistical parity. Here we build upon the area of auditing by exploring the multi-group setting under more complex definitions of fairness.
We propose a fairness-aware learning framework that mitigates intersectional subgroup bias associated with protected attributes. Prior research has primarily focused on mitigating one kind of bias by incorporating complex fairness-driven constraints into optimization objectives or designing additional layers that focus on specific protected attributes. We introduce a simple and generic bias mitigation approach that prevents models from learning relationships between protected attributes and output variable by reducing mutual information between them. We demonstrate that our approach is effecti…
Beyond Demographic Balance: Multi-Metric and Intersectional Evaluation of Fairness in MIMIC-IV Mortality Prediction
Virginia Tech（弗吉尼亚理工大学）；University of Texas at Arlington（德克萨斯大学阿灵顿分校）；University of Melbourne（墨尔本大学）；University of Toronto（多伦多大学）；University of Virginia（弗吉尼亚大学）；Bangladesh University of Engineering and Technology（孟加拉国工程技术大学）
Machine Learning with Multitype Protected Attributes: Intersectional Fairness through Regularisation
Centre for Actuarial Studies, Department of Economics, University of Melbourne, Australia（墨尔本大学经济系精算研究所以及中心）；Faculty of Economics and Business（经济与商业学院）；University of Amsterdam, The Netherlands（阿姆斯特丹大学）；LRisk, Leuven Research Center on Insurance（勒夫洛保险研究所以及中心）；LStat, Leuven Statistics Research Centre, KU Leuven, Belgium（勒夫洛统计研究所以及中心）
Carnegie Mellon University（卡内基梅隆大学）；University of Arizona（亚利桑那大学）
The fast spreading adoption of machine learning (ML) by companies across industries poses significant regulatory challenges. One such challenge is scalability: how can regulatory bodies efficiently audit these ML models, ensuring that they are fair? In this paper, we initiate the study of query-based auditing algorithms that can estimate the demographic parity of ML models in a query-efficient manner. We propose an optimal deterministic algorithm, as well as a practical randomized, oracle-efficient algorithm with comparable guarantees. Furthermore, we make inroads into understanding the optima…
In recent years, discussions about fairness in machine learning, AI ethics and algorithm audits have increased. Many entities have developed framework guidance to establish a baseline rubric for fairness and accountability. However, in spite of increased discussions and multiple frameworks, algorithm and data auditing still remain difficult to execute in practice. Many open-source auditing tools are available, but users aren't always aware of the tools, what they are useful for, or how to access them. Model auditing and evaluation are not frequently emphasized skills in machine learning. There…
German Aerospace Center (DLR)（德国航空航天中心）；Technical University of Munich (TUM)（慕尼黑工业大学）
Humanitarian mapping from space with machine learning helps policy-makers to timely and accurately identify people in need. However, recent concerns around fairness and transparency of algorithmic decision-making are a significant obstacle for applying these methods in practice. In this paper, we study if humanitarian mapping approaches from space are prone to bias in their predictions. We map village-level poverty and electricity rates in India based on nighttime lights (NTLs) with linear regression and random forest and analyze if the predictions systematically show prejudice against schedul…
University of California, Los Angeles（加利福尼亚大学洛杉矶分校）
Auditing machine learning-based (ML) healthcare tools for bias is critical to preventing patient harm, especially in communities that disproportionately face health inequities. General frameworks are becoming increasingly available to measure ML fairness gaps between groups. However, ML for health (ML4H) auditing principles call for a contextual, patient-centered approach to model assessment. Therefore, ML auditing tools must be (1) better aligned with ML4H auditing principles and (2) able to illuminate and characterize communities vulnerable to the most harm. To address this gap, we propose s…
