As AI-based decision systems proliferate, their successful operationalization requires balancing multiple desiderata: predictive performance, disparity across groups, safeguarding sensitive group attributes (e.g., race), and engineering cost. We present a holistic framework for evaluating and contextualizing fairness interventions with respect to the above desiderata. The two key points of practical consideration are \emph{where} (pre-, in-, post-processing) and \emph{how} (in what way the sensitive group data is used) the intervention is introduced. We demonstrate our framework with a case study on predictive parity. In it, we first propose a novel method for achieving predictive parity fairness without using group data at inference time via distibutionally robust optimization. Then, we showcase the effectiveness of these methods in a benchmarking study of close to 400 variations across two major model types (XGBoost vs. Neural Net), ten datasets, and over twenty unique methodologies. Methodological insights derived from our empirical study inform the practical design of ML workflow with fairness as a central concern. We find predictive parity is difficult to achieve without using group data, and despite requiring group data during model training (but not inference), distributionally robust methods we develop provide significant Pareto improvement. Moreover, a plain XGBoost model often Pareto-dominates neural networks with fairness interventions, highlighting the importance of model inductive bias.
Operationalizing Fairness: Post-Hoc Threshold Optimization Under Hard Resource Limits
Department of Computer Science and Engineering, DUIET, Dibrugarh University, Assam, India（迪布鲁大学计算机科学与工程系，DUIET，阿萨姆，印度）；Department of Computer Science and Engineering, MIT, Manipur University, 795003, India（曼尼普尔大学计算机科学与工程系，MIT，曼尼普尔大学，印度）
Balancing Performance and Fairness in Explainable AI for Anomaly Detection in Distributed Power Plants Monitoring
African Institute for Mathematical Sciences (AIMS)（非洲数学科学研究所）；Rhodes University（罗德斯大学）；National Institute for Theoretical and Computational Sciences (NITheCS)（国家理论与计算科学研究所）；Ludwig Maximilian University of Munich（慕尼黑路易斯·马克西米利安大学）；University of Ferrara（费拉拉大学）
What is Fair? Defining Fairness in Machine Learning for Health
Department of Statistical Sciences, University of Toronto（统计科学系，多伦多大学）；Harvard T.H. Chan School of Public Health（哈佛T.H. Chan公共卫生学院）；York University（约克大学）；Verily Life Sciences（Verily生命科学公司）；Duke University（杜克大学）；University of Toronto（多伦多大学）
Testing autonomous vehicles and AI: perspectives and challenges from cybersecurity, transparency, robustness and fairness
European Commission, Joint Research Centre（欧盟委员会联合研究中心）；University of Alcalá（阿尔卡拉大学）；EPFL - Swiss Federal Institute of Technology（瑞士洛桑联邦理工学院）；University of Oxford（牛津大学）；BASt - German Federal Highway Research Institute（德国联邦公路研究所）；Reed Mobility（里德移动公司）
Case Study: Predictive Fairness to Reduce Misdemeanor Recidivism Through Social Service Interventions
Carnegie Mellon University（卡内基梅隆大学）；Ctr for Data Science & Public Policy , University of Chicago（芝加哥大学数据科学与公共政策中心）；Los Angeles City Attorney’s Office（洛杉矶市检察官办公室）；Heinz College of Information Systems & Public Policy , Carnegie Mellon University（卡内基梅隆大学海因茨信息系统与公共政策学院）；Machine Learning Department , Carnegie Mellon University（卡内基梅隆大学机器学习系）；Director, Recidivism Reduction and Drug Diversion Unit (R2D2) , Los Angeles City Attorney’s Office（洛杉矶市检察官办公室再犯率降低和药物分流部门主任（R2D2））
The Explabox: Model-Agnostic Machine Learning Transparency & Analysis
Utrecht University（乌得勒支大学）；National Police Lab AI（国家警察人工智能实验室）；Netherlands National Police（荷兰国家警察）；Tilburg University（蒂尔堡大学）；Tilburg Institute for Law, Technology and Society（蒂尔堡法律、技术与社会研究所）
