Federated learning is an increasingly popular paradigm that enables a large number of entities to collaboratively learn better models. In this work, we study minimax group fairness in federated learning scenarios where different participating entities may only have access to a subset of the population groups during the training phase. We formally analyze how our proposed group fairness objective differs from existing federated learning fairness criteria that impose similar performance across participants instead of demographic groups. We provide an optimization algorithm -- FedMinMax -- for solving the proposed problem that provably enjoys the performance guarantees of centralized learning algorithms. We experimentally compare the proposed approach against other state-of-the-art methods in terms of group fairness in various federated learning setups, showing that our approach exhibits competitive or superior performance.
Fairness across different demographic groups is an essential criterion for face-related tasks, Face Attribute Classification (FAC) being a prominent example. Apart from this trend, Federated Learning (FL) is increasingly gaining traction as a scalable paradigm for distributed training. Existing FL approaches require data homogeneity to ensure fairness. However, this assumption is too restrictive in real-world settings. We propose F3, a novel FL framework for fair FAC under data heterogeneity. F3 adopts multiple heuristics to improve fairness across different demographic groups without requirin…
University of Pennsylvania（宾夕法尼亚大学）；Amazon AWS AI（亚马逊AWS人工智能部门）
We consider a recently introduced framework in which fairness is measured by worst-case outcomes across groups, rather than by the more standard differences between group outcomes. In this framework we provide provably convergent oracle-efficient learning algorithms (or equivalently, reductions to non-fair learning) for minimax group fairness. Here the goal is that of minimizing the maximum loss across all groups, rather than equalizing group losses. Our algorithms apply to both regression and classification settings and support both overall error and false positive or false negative rates as …
Grad2Fair: A Gradient-driven Approach for Graph Fairness without Demographics
School of Computer Science and Engineering, Sun Yat-Sen University（中山大学计算机科学与工程学院）；School of Software Engineering, Sun Yat-sen University（中山大学软件工程学院）；Institute of Artificial Intelligence, Xiamen University（厦门大学人工智能研究所）
Making fair decisions is crucial to ethically implementing machine learning algorithms in social settings. In this work, we consider the celebrated definition of counterfactual fairness [Kusner et al., NeurIPS, 2017]. We begin by showing that an algorithm which satisfies counterfactual fairness also satisfies demographic parity, a far simpler fairness constraint. Similarly, we show that all algorithms satisfying demographic parity can be trivially modified to satisfy counterfactual fairness. Together, our results indicate that counterfactual fairness is basically equivalent to demographic pari…
Fairness without Demographics through Learning Graph of Gradients
Carnegie Mellon University（卡内基梅隆大学）；The Chinese University of Hong Kong（香港中文大学）；Institute of Automation, Chinese Academy of Sciences（中国科学院自动化研究所）；Tsinghua University（清华大学）
Heinz College of Information Systems and Public Policy, Carnegie Mellon University（卡内基梅隆大学海因茨信息系统与公共政策学院）
We propose a general approach for training survival analysis models that minimizes a worst-case error across all subpopulations that are large enough (occurring with at least a user-specified minimum probability). This approach uses a training loss function that does not know any demographic information to treat as sensitive. Despite this, we demonstrate that our proposed approach often scores better on recently established fairness metrics (without a significant drop in prediction accuracy) compared to various baselines, including ones which directly use sensitive demographic information in t…
In this work we formulate and formally characterize group fairness as a multi-objective optimization problem, where each sensitive group risk is a separate objective. We propose a fairness criterion where a classifier achieves minimax risk and is Pareto-efficient w.r.t. all groups, avoiding unnecessary harm, and can lead to the best zero-gap model if policy dictates so. We provide a simple optimization algorithm compatible with deep neural networks to satisfy these constraints. Since our method does not require test-time access to sensitive attributes, it can be applied to reduce worst-case cl…
Partnership on AI（人工智能伙伴关系）；Minnesota State University, Mankato（明尼苏达州立大学曼卡托分校）；Sony AI（索尼人工智能）
As calls for fair and unbiased algorithmic systems increase, so too does the number of individuals working on algorithmic fairness in industry. However, these practitioners often do not have access to the demographic data they feel they need to detect bias in practice. Even with the growing variety of toolkits and strategies for working towards algorithmic fairness, they almost invariably require access to demographic attributes or proxies. We investigated this dilemma through semi-structured interviews with 38 practitioners and professionals either working in or adjacent to algorithmic fairne…
Beyond Demographic Balance: Multi-Metric and Intersectional Evaluation of Fairness in MIMIC-IV Mortality Prediction
Virginia Tech（弗吉尼亚理工大学）；University of Texas at Arlington（德克萨斯大学阿灵顿分校）；University of Melbourne（墨尔本大学）；University of Toronto（多伦多大学）；University of Virginia（弗吉尼亚大学）；Bangladesh University of Engineering and Technology（孟加拉国工程技术大学）
On Demographic Group Fairness Guarantees in Deep Learning
Harvard AI and Robotics Lab at Harvard University（哈佛大学人工智能与机器人实验室）；Embodied AI and Robotics (AIR) Lab at New York University（纽约大学具身人工智能与机器人实验室）；School of Computing and Informatics, University of Louisiana at Lafayette（路易斯安那州立大学拉法叶分校计算机与信息学院）
The use of machine learning models in consequential decision making often exacerbates societal inequity, in particular yielding disparate impact on members of marginalized groups defined by race and gender. The area under the ROC curve (AUC) is widely used to evaluate the performance of a scoring function in machine learning, but is studied in algorithmic fairness less than other performance metrics. Due to the pairwise nature of the AUC, defining an AUC-based group fairness metric is pairwise-dependent and may involve both \emph{intra-group} and \emph{inter-group} AUCs. Importantly, consideri…
Max Planck Institute for Informatics（马克斯·普朗克信息学研究所）；Google Research（谷歌研究院）
Much of the previous machine learning (ML) fairness literature assumes that protected features such as race and sex are present in the dataset, and relies upon them to mitigate fairness concerns. However, in practice factors like privacy and regulation often preclude the collection of protected features, or their use for training or inference, severely limiting the applicability of traditional fairness research. Therefore we ask: How can we train an ML model to improve fairness when we do not even know the protected group memberships? In this work we address this problem by proposing Adversari…
Towards Harmless Rawlsian Fairness Regardless of Demographic Prior
School of Computer Science and Engineering, Beihang University（北京航空航天大学计算机科学与工程学院）；Institute of High-Performance Computing, Agency for Science, Technology and Research（新加坡科技研究局高性能计算研究所）；Centre for Frontier AI Research, Agency for Science, Technology and Research（新加坡科技研究局前沿人工智能研究中心）；College of Computing and Data Science, Nanyang Technological University（南洋理工大学计算与数据科学学院）
When dealing with sensitive data in automated data-driven decision-making, an important concern is to learn predictors with high performance towards a class label, whilst minimising for the discrimination towards any sensitive attribute, like gender or race, induced from biased data. A few hybrid tree optimisation criteria exist that combine classification performance and fairness. Although the threshold-free ROC-AUC is the standard for measuring traditional classification model performance, current fair tree classification methods mainly optimise for a fixed threshold on both the classificati…
Observed events in recommendation are consequence of the decisions made by a policy, thus they are usually selectively labeled, namely the data are Missing Not At Random (MNAR), which often causes large bias to the estimate of true outcomes risk. A general approach to correct MNAR bias is performing small Randomized Controlled Trials (RCTs), where an additional uniform policy is employed to randomly assign items to each user. In this work, we concentrate on the fairness of RCTs under both homogeneous and heterogeneous demographics, especially analyzing the bias for the least favorable group on…
Improving Fairness of Large Language Model-Based ICU Mortality Prediction via Case-Based Prompting
School of Computer Science and Engineering, Tianjin University of Technology, Tianjin, China（天津理工大学计算机科学与工程学院）；Institute for Artificial Intelligence, Peking University, Beijing, China（北京大学人工智能研究院）；DCST, BNRist, RIIT, Institute of Internet Industry, Tsinghua University, Beijing, China（清华大学信息产业研究院）；National Institute of Health Data Science, Peking University, Beijing, China（北京大学健康数据科学国家研究院）
There has been increasing awareness of ethical issues in machine learning, and fairness has become an important research topic. Most fairness efforts in computer vision have been focused on human sensing applications and preventing discrimination by people's physical attributes such as race, skin color or age by increasing visual representation for particular demographic groups. We argue that ML fairness efforts should extend to object recognition as well. Buildings, artwork, food and clothing are examples of the objects that define human culture. Representing these objects fairly in machine l…
Classifiers that achieve demographic balance by explicitly using protected attributes such as race or gender are often politically or culturally controversial due to their lack of individual fairness, i.e. individuals with similar qualifications will receive different outcomes. Individually and group fair decision criteria can produce counter-intuitive results, e.g. that the optimal constrained boundary may reject intuitively better candidates due to demographic imbalance in similar candidates. Both approaches can be seen as introducing individual resentment, where some individuals would have …
FairFedMed: Benchmarking Group Fairness in Federated Medical Imaging with FairLoRA
Harvard AI and Robotics Lab（哈佛人工智能与机器人实验室）；Harvard University（哈佛大学）；Harvard Ophthalmology AI lab（哈佛眼科人工智能实验室）；University of Central Florida（佛罗里达中央大学）；University of Louisiana at Lafayette（路易斯安那州立大学拉斐特分校）；New York University（纽约大学）；NYUAD Center for Artificial Intelligence and Robotics（纽约大学阿布扎克人工智能与机器人中心）
Safe Fairness Guarantees Without Demographics in Classification: Spectral Uncertainty Set Perspective
Basque Center for Applied Mathematics (BCAM)（巴斯克应用数学中心）；Predictive Analytics Lab, University of Sussex（预测分析实验室，苏塞克斯大学）；Monash University Indonesia（墨尔本大学印尼分校）；University of the Basque Country EHU（巴斯克国家大学EHU）
Demographic classification is essential in fairness assessment in recommender systems or in measuring unintended bias in online networks and voting systems. Important fields like education and politics, which often lay a foundation for the future of equality in society, need scrutiny to design policies that can better foster equality in resource distribution constrained by the unbalanced demographic distribution of people in the country. We collect three publicly available datasets to train state-of-the-art classifiers in the domain of gender and caste classification. We train the models in th…
Are demographically invariant models and representations in medical imaging fair?
Technical University of Denmark（丹麦技术大学）；Pioneer Centre for AI（人工智能先锋中心）；Research Institute for Signals, Systems and Computational Intelligence (CONICET)（信号、系统与计算智能研究所（CONICET））；Universidad Nacional del Litoral（国立 littoral 大学）；University of Copenhagen（哥本哈根大学）；Neurobiology Research Unit, Rigshospitalet（哥本哈根大学医院神经生物学研究单元）
