Kearns et al. [2018] recently proposed a notion of rich subgroup fairness intended to bridge the gap between statistical and individual notions of fairness. Rich subgroup fairness picks a statistical fairness constraint (say, equalizing false positive rates across protected groups), but then asks that this constraint hold over an exponentially or infinitely large collection of subgroups defined by a class of functions with bounded VC dimension. They give an algorithm guaranteed to learn subject to this constraint, under the condition that it has access to oracles for perfectly learning absent a fairness constraint. In this paper, we undertake an extensive empirical evaluation of the algorithm of Kearns et al. On four real datasets for which fairness is a concern, we investigate the basic convergence of the algorithm when instantiated with fast heuristics in place of learning oracles, measure the tradeoffs between fairness and accuracy, and compare this approach with the recent algorithm of Agarwal et al. [2018], which implements weaker and more traditional marginal fairness constraints defined by individual protected attributes. We find that in general, the Kearns et al. algorithm converges quickly, large gains in fairness can be obtained with mild costs to accuracy, and that optimizing accuracy subject only to marginal fairness leads to classifiers with substantial subgroup unfairness. We also provide a number of analyses and visualizations of the dynamics and behavior of the Kearns et al. algorithm. Overall we find this algorithm to be effective on real data, and rich subgroup fairness to be a viable notion in practice.
McGill University（麦吉尔大学）；Mila, Quebec AI Institute（米拉魁北克人工智能研究所）；Université Laval（拉瓦尔大学）；University of Western Ontario（西安大略大学）
We propose an analysis in fair learning that preserves the utility of the data while reducing prediction disparities under the criteria of group sufficiency. We focus on the scenario where the data contains multiple or even many subgroups, each with limited number of samples. As a result, we present a principled method for learning a fair predictor for all subgroups via formulating it as a bilevel objective. Specifically, the subgroup specific predictors are learned in the lower-level through a small amount of data and the fair predictor. In the upper-level, the fair predictor is updated to be…
Courant Institute of Mathematical Sciences, New York University（纽约大学柯朗数学科学研究所）；Google AI（谷歌人工智能）
The potential for learned models to amplify existing societal biases has been broadly recognized. Fairness-aware classifier constraints, which apply equality metrics of performance across subgroups defined on sensitive attributes such as race and gender, seek to rectify inequity but can yield non-uniform degradation in performance for skewed datasets. In certain domains, imbalanced degradation of performance can yield another form of unintentional bias. In the spirit of constructing fairness-aware algorithms as societal imperative, we explore an alternative: Pareto-Efficient Fairness (PEF). Th…
Massachusetts Institute of Technology（麻省理工学院）；MIT-IBM Watson AI Lab（麻省理工学院-IBM沃森人工智能实验室）；IBM Research（IBM研究院）
Selective regression allows abstention from prediction if the confidence to make an accurate prediction is not sufficient. In general, by allowing a reject option, one expects the performance of a regression model to increase at the cost of reducing coverage (i.e., by predicting on fewer samples). However, as we show, in some cases, the performance of a minority subgroup can decrease while we reduce the coverage, and thus selective regression can magnify disparities between different sensitive subgroups. Motivated by these disparities, we propose new fairness criteria for selective regression …
With the aim of building machine learning systems that incorporate standards of fairness and accountability, we explore explicit subgroup sample complexity bounds. The work is motivated by the observation that classifier predictions for real world datasets often demonstrate drastically different metrics, such as accuracy, when subdivided by specific sensitive variable subgroups. The reasons for these discrepancies are varied and not limited to the influence of mitigating variables, institutional bias, underlying population distributions as well as sampling bias. Among the numerous definitions …
University of Toronto（多伦多大学）；Vector Institute（向量研究所）；University of Tübingen（蒂宾根大学）；Massachusetts Institute of Technology（麻省理工学院）
Deep metric learning (DML) enables learning with less supervision through its emphasis on the similarity structure of representations. There has been much work on improving generalization of DML in settings like zero-shot retrieval, but little is known about its implications for fairness. In this paper, we are the first to evaluate state-of-the-art DML methods trained on imbalanced data, and to show the negative impact these representations have on minority subgroup performance when used for downstream tasks. In this work, we first define fairness in DML through an analysis of three properties…
Toyota Technological Institute at Chicago（芝加哥丰田技术研究所）；Microsoft Research（微软研究院）；Cornell University（康奈尔大学）
We study methods for improving fairness to subgroups in settings with overlapping populations and sequential predictions. Classical notions of fairness focus on the balance of some property across different populations. However, in many applications the goal of the different groups is not to be predicted equally but rather to be predicted well. We demonstrate that the task of satisfying this guarantee for multiple overlapping groups is not straightforward and show that for the simple objective of unweighted average of false negative and false positive rate, satisfying this for overlapping popu…
University of Pennsylvania（宾夕法尼亚大学）；Microsoft Research-New York City（微软研究院（纽约市））
The most prevalent notions of fairness in machine learning are statistical definitions: they fix a small collection of pre-defined groups, and then ask for parity of some statistic of the classifier across these groups. Constraints of this form are susceptible to intentional or inadvertent "fairness gerrymandering", in which a classifier appears to be fair on each individual group, but badly violates the fairness constraint on one or more structured subgroups defined over the protected attributes. We propose instead to demand statistical notions of fairness across exponentially (or infinitely)…
Despite enormous successful applications of graph neural networks (GNNs), theoretical understanding of their generalization ability, especially for node-level tasks where data are not independent and identically-distributed (IID), has been sparse. The theoretical investigation of the generalization performance is beneficial for understanding fundamental issues (such as fairness) of GNN models and designing better learning methods. In this paper, we present a novel PAC-Bayesian analysis for GNNs under a non-IID semi-supervised learning setup. Moreover, we analyze the generalization performances…
The issue of fairness arises when the automatic speech recognition (ASR) systems do not perform equally well for all subgroups of the population. In any fairness measurement studies for ASR, the open questions of how to control the nuisance factors, how to handle unobserved heterogeneity across speakers, and how to trace the source of any word error rate (WER) gap among different subgroups are especially important - if not appropriately accounted for, incorrect conclusions will be drawn. In this paper, we introduce mixed-effects Poisson regression to better measure and interpret any WER differ…
Prediction systems are successfully deployed in applications ranging from disease diagnosis, to predicting credit worthiness, to image recognition. Even when the overall accuracy is high, these systems may exhibit systematic biases that harm specific subpopulations; such biases may arise inadvertently due to underrepresentation in the data used to train a machine-learning model, or as the result of intentional malicious discrimination. We develop a rigorous framework of *multiaccuracy* auditing and post-processing to ensure accurate predictions across *identifiable subgroups*. Our algorithm, M…
University College Dublin（都柏林大学学院）；Czech Technical University in Prague（布拉格捷克理工大学）；Imperial College London（帝国理工学院）
In machine learning, training data often capture the behaviour of multiple subgroups of some underlying human population. When the amounts of training data for the subgroups are not controlled carefully, under-representation bias arises. We introduce two natural notions of subgroup fairness and instantaneous fairness to address such under-representation bias in time-series forecasting problems. In particular, we consider the subgroup-fair and instant-fair learning of a linear dynamical system (LDS) from multiple trajectories of varying lengths, and the associated forecasting problems. We provi…
Beyond Demographic Balance: Multi-Metric and Intersectional Evaluation of Fairness in MIMIC-IV Mortality Prediction
Virginia Tech（弗吉尼亚理工大学）；University of Texas at Arlington（德克萨斯大学阿灵顿分校）；University of Melbourne（墨尔本大学）；University of Toronto（多伦多大学）；University of Virginia（弗吉尼亚大学）；Bangladesh University of Engineering and Technology（孟加拉国工程技术大学）
Looking Beyond What You See: An Empirical Analysis on Subgroup Intersectional Fairness for Multi-label Chest X-ray Classification Using Social Determinants of Racial Health Inequities
Massachusetts Institute of Technology（麻省理工学院）；University at Buffalo（布法罗大学）
The growing capability and accessibility of machine learning has led to its application to many real-world domains and data about people. Despite the benefits algorithmic systems may bring, models can reflect, inject, or exacerbate implicit and explicit societal biases into their outputs, disadvantaging certain demographic subgroups. Discovering which biases a machine learning model has introduced is a great challenge, due to the numerous definitions of fairness and the large number of potentially impacted subgroups. We present FairVis, a mixed-initiative visual analytics system that integrate…
Nanyang Technological University（南洋理工大学）；Institute of High Performance Computing (IHPC), Agency for Science, Technology and Research（新加坡科技研究局高性能计算研究所（IHPC））；Centre for Frontier AI Research (CFAR), Agency for Science, Technology and Research（新加坡科技研究局前沿人工智能研究中心（CFAR））；Beihang University（北京航空航天大学）；New Jersey Institute of Technology（新泽西理工学院）；Zhejiang Sci-Tech University（浙江理工大学）
Understanding challenges to the interpretation of disaggregated evaluations of algorithmic fairness
Google Research, Mountain View, CA, USA（谷歌研究, 山景城, 加州, 美国）；Google DeepMind, Mountain View, CA, USA（谷歌深Mind, 山景城, 加州, 美国）；New York University, New York, NY, USA（纽约大学, 纽约, 纽约, 美国）；Massachusetts Institute of Technology, Cambridge, MA, USA（麻省理工学院, 柏林, 马萨诸塞, 美国）；The Hopsital for Sick Children, Toronto, ON, Canada（儿童医院, 温哥华, 加拿大）；Stanford University, Stanford, CA, USA（斯坦福大学, 斯坦福, 加州, 美国）
Algorithmic fairness and privacy are essential pillars of trustworthy machine learning. Fair machine learning aims at minimizing discrimination against protected groups by, for example, imposing a constraint on models to equalize their behavior across different groups. This can subsequently change the influence of training data points on the fair model, in a disproportionate way. We study how this can change the information leakage of the model about its training data. We analyze the privacy risks of group fairness (e.g., equalized odds) through the lens of membership inference attacks: inferr…
Improving Fairness of Large Language Model-Based ICU Mortality Prediction via Case-Based Prompting
School of Computer Science and Engineering, Tianjin University of Technology, Tianjin, China（天津理工大学计算机科学与工程学院）；Institute for Artificial Intelligence, Peking University, Beijing, China（北京大学人工智能研究院）；DCST, BNRist, RIIT, Institute of Internet Industry, Tsinghua University, Beijing, China（清华大学信息产业研究院）；National Institute of Health Data Science, Peking University, Beijing, China（北京大学健康数据科学国家研究院）
As the use of deep learning in high impact domains becomes ubiquitous, it is increasingly important to assess the resilience of models. One such high impact domain is that of face recognition, with real world applications involving images affected by various degradations, such as motion blur or high exposure. Moreover, images captured across different attributes, such as gender and race, can also challenge the robustness of a face recognition algorithm. While traditional summary statistics suggest that the aggregate performance of face recognition models has continued to improve, these metrics…
Disease identification is a core, routine activity in observational health research. Cohorts impact downstream analyses, such as how a condition is characterized, how patient risk is defined, and what treatments are studied. It is thus critical to ensure that selected cohorts are representative of all patients, independently of their demographics or social determinants of health. While there are multiple potential sources of bias when constructing phenotype definitions which may affect their fairness, it is not standard in the field of phenotyping to consider the impact of different definition…
Ethical Fairness in Ubiquitous Health Sensing without Known Attributes
Ira A. Fulton Schools of Engineering, Arizona State University（亚利桑那州立大学弗里曼工程学院）；Arizona State University（亚利桑那州立大学）；Cornell University（康奈尔大学）；University of Michigan（密歇根大学）
Toward Fairness in Machine Learning Models for Predicting Treatment Retention and Premature Discontinuation in Medication for Opioid Use Disorder
Gary W. Rollins College of Business, The University of Tennessee at Chattanooga（田纳西大学查塔努加分校加里·W·罗林斯商学院）；The University of Texas at San Antonio（德克萨斯大学圣安东尼奥分校）；The University of Texas Health Science Center at Houston（德克萨斯大学休斯顿健康科学中心）；Texas A&M University（德克萨斯A&M大学）
