Debiasing classifiers: is reality at variance with expectation?
Birla Institute of Technology and Science（贝拉理工学院）；Ludwig-Maximilians-University（路德维希-马克西米利安大学）；J.P. Morgan（摩根大通）；J.P. Morgan AI Research（摩根大通人工智能研究院）；University of Warwick（华威大学）
We present an empirical study of debiasing methods for classifiers, showing that debiasers often fail in practice to generalize out-of-sample, and can in fact make fairness worse rather than better. A rigorous evaluation of the debiasing treatment effect requires extensive cross-validation beyond what is usually done. We demonstrate that this phenomenon can be explained as a consequence of bias-variance trade-off, with an increase in variance necessitated by imposing a fairness constraint. Follow-up experiments validate the theoretical prediction that the estimation variance depends strongly on the base rates of the protected class. Considering fairness--performance trade-offs justifies the counterintuitive notion that partial debiasing can actually yield better results in practice on out-of-sample data.
University of Rochester（罗切斯特大学）；Kitware, Inc.（Kitware公司）
Deep image classifiers have been found to learn biases from datasets. To mitigate the biases, most previous methods require labels of protected attributes (e.g., age, skin tone) as full-supervision, which has two limitations: 1) it is infeasible when the labels are unavailable; 2) they are incapable of mitigating unknown biases -- biases that humans do not preconceive. To resolve those problems, we propose Debiasing Alternate Networks (DebiAN), which comprises two networks -- a Discoverer and a Classifier. By training in an alternate manner, the discoverer tries to find multiple unknown biases…
Deep neural networks for image-based screening and computer-aided diagnosis have achieved expert-level performance on various medical imaging modalities, including chest radiographs. Recently, several works have indicated that these state-of-the-art classifiers can be biased with respect to sensitive patient attributes, such as race or gender, leading to growing concerns about demographic disparities and discrimination resulting from algorithmic and model-based decision-making in healthcare. Fair machine learning has focused on mitigating such biases against disadvantaged or marginalised group…
Dictionary-based Pathology Mining with Hard-instance-assisted Classifier Debiasing for Genetic Biomarker Prediction from WSIs
Shanghai Key Laboratory of Multidimensional Information Processing, East China Normal University（华东师范大学上海市多维度信息处理重点实验室）；Department of General Surgery, Tongji Hospital, Tongji University School of Medicine（同济大学医学院附属同济医院普外科）
Machine learning fairness concerns about the biases towards certain protected or sensitive group of people when addressing the target tasks. This paper studies the debiasing problem in the context of image classification tasks. Our data analysis on facial attribute recognition demonstrates (1) the attribution of model bias from imbalanced training data distribution and (2) the potential of adversarial examples in balancing data distribution. We are thus motivated to employ adversarial example to augment the training data for visual debiasing. Specifically, to ensure the adversarial generalizat…
Freie Universität Berlin（柏林自由大学）；University of Southern Denmark（南丹麦大学）；Universität der Bundeswehr München（慕尼黑联邦国防军大学）
Hate speech detection is a common downstream application of natural language processing (NLP) in the real world. In spite of the increasing accuracy, current data-driven approaches could easily learn biases from the imbalanced data distributions originating from humans. The deployment of biased models could further enhance the existing social biases. But unlike handling tabular data, defining and mitigating biases in text classifiers, which deal with unstructured data, are more challenging. A popular solution for improving machine learning fairness in NLP is to conduct the debiasing process wi…
McGill University（麦吉尔大学）；Amirkabir University of Technology (Tehran Polytechnic)（阿米尔卡比尔理工大学（德黑兰理工））；Islamic Azad University（伊斯兰阿扎德大学）
Background: Virtual reality simulators and machine learning have the potential to augment understanding, assessment and training of psychomotor performance in neurosurgery residents. Objective: This study outlines the first application of machine learning to distinguish "skilled" and "novice" psychomotor performance during a virtual reality neurosurgical task. Methods: Twenty-three neurosurgeons and senior neurosurgery residents comprising the "skilled" group and 92 junior neurosurgery residents and medical students the "novice" group. The task involved removing a series of virtual brain tumor…
Floyd B. Buchanan High School（弗洛伊德·B·布坎南高中）；Massachusetts Institute of Technology（麻省理工学院）
Machine learning (ML) models are increasingly used for high-stake applications that can greatly impact people's lives. Despite their use, these models have the potential to be biased towards certain social groups on the basis of race, gender, or ethnicity. Many prior works have attempted to mitigate this "model discrimination" by updating the training data (pre-processing), altering the model learning process (in-processing), or manipulating model output (post-processing). However, these works have not yet been extended to the realm of multi-sensitive parameters and sensitive options (MSPSO), …
ICSI（国际计算机科学研究所）；UC Berkeley（加州大学伯克利分校）；University of Jena（耶拿大学）；UMass Lowell（马萨诸塞大学洛厄尔分校）
Images seen during test time are often not from the same distribution as images used for learning. This problem, known as domain shift, occurs when training classifiers from object-centric internet image databases and trying to apply them directly to scene understanding tasks. The consequence is often severe performance degradation and is one of the major barriers for the application of classifiers in real-world systems. In this paper, we show how to learn transform-based domain adaptation classifiers in a scalable manner. The key idea is to exploit an implicit rank constraint, originated from…
AdapFair: Ensuring Adaptive Fairness for Machine Learning Operations
Department of Information Systems and Intelligent Business, School of Management, Xi’an Jiaotong University（信息系统与智能商业系，管理学院，西安交通大学）；School of Statistics and Data Science, Shanghai University of Finance and Economics（统计与数据科学学院，上海财经大学）
Duke University School of Medicine（杜克大学医学院）；Duke University（杜克大学）；National Artificial Intelligence Institute, U.S. Dept. of Veterans Affairs（美国退伍军人事务部国家人工智能研究所）
Research studies of artificial intelligence models in medical imaging have been hampered by poor generalization. This problem has been especially concerning over the last year with numerous applications of deep learning for COVID-19 diagnosis. Virtual imaging trials (VITs) could provide a solution for objective evaluation of these models. In this work utilizing the VITs, we created the CVIT-COVID dataset including 180 virtually imaged computed tomography (CT) images from simulated COVID-19 and normal phantom models under different COVID-19 morphology and imaging properties. We evaluated the pe…
Fairness-aware machine learning seeks to maximise utility in generating predictions while avoiding unfair discrimination based on sensitive attributes such as race, sex, religion, etc. An important line of work in this field is enforcing fairness during the training step of a classifier. A simple yet effective binary classification algorithm that follows this strategy is two-naive-Bayes (2NB), which enforces statistical parity - requiring that the groups comprising the dataset receive positive labels with the same likelihood. In this paper, we generalise this algorithm into N-naive-Bayes (NNB)…
Helmholtz AI（亥姆霍兹AI）；Ecole Polytechnique Fédérale de Lausanne（洛桑联邦理工学院）；Swisscom（瑞士电信）
The goal of fairness in classification is to learn a classifier that does not discriminate against groups of individuals based on sensitive attributes, such as race and gender. One approach to designing fair algorithms is to use relaxations of fairness notions as regularization terms or in a constrained optimization problem. We observe that the hyperbolic tangent function can approximate the indicator function. We leverage this property to define a differentiable relaxation that approximates fairness notions provably better than existing relaxations. In addition, we propose a model-agnostic mu…
