Diagnosing failures of fairness transfer across distribution shift in real-world medical settings
Google Research（谷歌研究院）；University of Basel（巴塞尔大学）；DeepMind（深度思维）
Jessica Schrouff, Natalie Harris, Oluwasanmi Koyejo, Ibrahim Alabdulmohsin, Eva Schnider, Krista Opsahl-Ong, Alex Brown, Subhrajit Roy, Diana Mincu, Christina C… 展开作者
Jessica Schrouff, Natalie Harris, Oluwasanmi Koyejo, Ibrahim Alabdulmohsin, Eva Schnider, Krista Opsahl-Ong, Alex Brown, Subhrajit Roy, Diana Mincu, Christina Chen, Awa Dieng, Yuan Liu, Vivek Natarajan, Alan Karthikesalingam, Katherine Heller, Silvia Chiappa, Alexander D'Amour
发表信息：
Advances in Neural Information Processing Systems 35 (NeurIPS 2022)
Diagnosing and mitigating changes in model fairness under distribution shift is an important component of the safe deployment of machine learning in healthcare settings. Importantly, the success of any mitigation strategy strongly depends on the structure of the shift. Despite this, there has been little discussion of how to empirically assess the structure of a distribution shift that one is encountering in practice. In this work, we adopt a causal framing to motivate conditional independence tests as a key tool for characterizing distribution shifts. Using our approach in two medical applications, we show that this knowledge can help diagnose failures of fairness transfer, including cases where real-world shifts are more complex than is often assumed in the literature. Based on these results, we discuss potential remedies at each step of the machine learning pipeline.
Centre for Intelligent Machines, McGill University（麦吉尔大学智能机器中心）
Although deep learning (DL) models have shown great success in many medical image analysis tasks, deployment of the resulting models into real clinical contexts requires: (1) that they exhibit robustness and fairness across different sub-populations, and (2) that the confidence in DL model predictions be accurately expressed in the form of uncertainties. Unfortunately, recent studies have indeed shown significant biases in DL models across demographic subgroups (e.g., race, sex, age) in the context of medical image analysis, indicating a lack of fairness in the models. Although several methods…
The Boundaries of Fair AI in Medical Image Prognosis: A Causal Perspective
Department of Computer Science and Engineering, The Ohio State University（计算机科学与工程系，俄亥俄州立大学）；Department of Biomedical Informatics, The Ohio State University（生物医学信息学系，俄亥俄州立大学）；Department of Ophthalmology and Visual Sciences, The Ohio State University（眼科学与视觉科学系，俄亥俄州立大学）
Georgia State University（佐治亚州立大学）；Georgia Institute of Technology（佐治亚理工学院）；Institute of Epidemiology, Disease Control and Research（流行病学、疾病控制与研究所）
Effective and reliable screening of patients via Computer-Aided Diagnosis can play a crucial part in the battle against COVID-19. Most of the existing works focus on developing sophisticated methods yielding high detection performance, yet not addressing the issue of predictive uncertainty. In this work, we introduce uncertainty estimation to detect confusing cases for expert referral to address the unreliability of state-of-the-art (SOTA) DNNs on COVID-19 detection. To the best of our knowledge, we are the first to address this issue on the COVID-19 detection problem. In this work, we investi…
Towards unraveling calibration biases in medical image analysis
Hospital Italiano de Buenos Aires（布宜诺斯艾利斯意大利医院）；Universidad Tecnológica Nacional（国立技术大学）；Research Institute for Signals, Systems and Computational Intelligence sinc(i) (FICH-UNL/CONICET)（信号、系统与计算智能研究所sinc(i)（FICH-UNL/阿根廷国家科学与技术研究理事会））
Decision makers increasingly rely on algorithmic risk scores to determine access to binary treatments including bail, loans, and medical interventions. In these settings, we reconcile two fairness criteria that were previously shown to be in conflict: calibration and error rate equality. In particular, we derive necessary and sufficient conditions for the existence of calibrated scores that yield classifications achieving equal error rates at any given group-blind threshold. We then present an algorithm that searches for the most accurate score subject to both calibration and minimal error rat…
University of Cambridge（剑桥大学）；The Alan Turing Institute（艾伦·图灵研究所）
We investigate the influence of adversarial training on the interpretability of convolutional neural networks (CNNs), specifically applied to diagnosing skin cancer. We show that gradient-based saliency maps of adversarially trained CNNs are significantly sharper and more visually coherent than those of standardly trained CNNs. Furthermore, we show that adversarially trained networks highlight regions with significant color variation within the lesion, a common characteristic of melanoma. We find that fine-tuning a robust network with a small learning rate further improves saliency maps' sharp…
Islamic Azad University of Gorgan Branch（戈尔甘伊斯兰阿扎德大学分校）
Today data mining techniques are exploited in medical science for diagnosing, overcoming and treating diseases. Neural network is one of the techniques which are widely used for diagnosis in medical field. In this article efficiency of nine algorithms, which are basis of neural network learning in diagnosing cardiovascular diseases, will be assessed. Algorithms are assessed in terms of accuracy, sensitivity, transparency, AROC and convergence rate by means of 10 fold cross validation. The results suggest that in training phase, Lonberg-M algorithm has the best efficiency in terms of all metric…
FairFedMed: Benchmarking Group Fairness in Federated Medical Imaging with FairLoRA
Harvard AI and Robotics Lab（哈佛人工智能与机器人实验室）；Harvard University（哈佛大学）；Harvard Ophthalmology AI lab（哈佛眼科人工智能实验室）；University of Central Florida（佛罗里达中央大学）；University of Louisiana at Lafayette（路易斯安那州立大学拉斐特分校）；New York University（纽约大学）；NYUAD Center for Artificial Intelligence and Robotics（纽约大学阿布扎克人工智能与机器人中心）
Fairness in Multi-modal Medical Diagnosis with Demonstration Selection
Arizona State University（亚利桑那州立大学）；University of Rochester（罗切斯特大学）；University of Virginia（弗吉尼亚大学）；UCL（伦敦大学学院）；University of North Carolina at Chapel Hill（北卡罗来纳大学教堂山分校）；University of Central Florida（佛罗里达中央大学）
The method of microwave radiometry is one of the areas of medical diagnosis of breast cancer. It is based on analysis of the spatial distribution of internal and surface tissue temperatures, which are measured in the microwave (RTM) and infrared (IR) ranges. Complex mathematical and computer models describing complex physical and biological processes within biotissue increase the efficiency of this method. Physical and biological processes are related to temperature dynamics and microwave electromagnetic radiation. Verification and validation of the numerical model is a key challenge to ensure…
University of Edinburgh（爱丁堡大学）；Samsung AI Centre, Cambridge（三星剑桥人工智能中心）
A multitude of work has shown that machine learning-based medical diagnosis systems can be biased against certain subgroups of people. This has motivated a growing number of bias mitigation algorithms that aim to address fairness issues in machine learning. However, it is difficult to compare their effectiveness in medical imaging for two reasons. First, there is little consensus on the criteria to assess fairness. Second, existing bias mitigation algorithms are developed under different settings, e.g., datasets, model selection strategies, backbones, and fairness metrics, making a direct comp…
Towards Trustworthy Audio Deepfake Detection: A Systematic Framework for Diagnosing and Mitigating Gender Bias
School of Computing, Wichita State University（维克托利亚州立大学计算机学院）；Institut national de la recherche scientifique (INRS-EMT)（国家科学研究机构（INRS-EMT））；INRS-UQO Mixed Research Unit on Cybersecurity（INRS-UQO网络安全混合研究单位）
The University of British Columbia（不列颠哥伦比亚大学）；The Chinese University of Hong Kong（香港中文大学）
Mitigating the discrimination of machine learning models has gained increasing attention in medical image analysis. However, rare works focus on fair treatments for patients with multiple sensitive demographic ones, which is a crucial yet challenging problem for real-world clinical applications. In this paper, we propose a novel method for fair representation learning with respect to multi-sensitive attributes. We pursue the independence between target and multi-sensitive representations by achieving orthogonality in the representation space. Concretely, we enforce the column space orthogonali…
University of Connecticut（康涅狄格大学）；AuCoDe；Universidade Federal de Minas Gerais（米纳斯吉拉斯联邦大学）；Meedan；University of Washington School of Medicine（华盛顿大学医学院）；Michigan State University College of Human Medicine（密歇根州立大学人类医学院）
Most Fairness in AI research focuses on exposing biases in AI systems. A broader lens on fairness reveals that AI can serve a greater aspiration: rooting out societal inequities from their source. Specifically, we focus on inequities in health information, and aim to reduce bias in that domain using AI. The AI algorithms under the hood of search engines and social media, many of which are based on recommender systems, have an outsized impact on the quality of medical and health information online. Therefore, embedding bias detection and reduction into these recommender systems serving up medic…
Diagnosing Shoulder Disorders Using Multimodal Large Language Models and Consumer-Grade Cameras
Bytedance（字节跳动）；Peking University（北京大学）；Peking University People’s Hospital（北京大学人民医院）；Mohamed bin Zayed University of Artificial Intelligence（穆罕默德·本·扎耶德人工智能大学）；The Chinese University of Hong Kong（香港中文大学）
Are demographically invariant models and representations in medical imaging fair?
Technical University of Denmark（丹麦技术大学）；Pioneer Centre for AI（人工智能先锋中心）；Research Institute for Signals, Systems and Computational Intelligence (CONICET)（信号、系统与计算智能研究所（CONICET））；Universidad Nacional del Litoral（国立 littoral 大学）；University of Copenhagen（哥本哈根大学）；Neurobiology Research Unit, Rigshospitalet（哥本哈根大学医院神经生物学研究单元）
Fairness-Aware Fine-Tuning of Vision-Language Models for Medical Glaucoma Diagnosis
Department of Computer Science, University of Rochester, NY, USA（罗切斯特大学计算机科学系）；Biostatistics and Health Data Science, School of Medicine, Indiana University, Indianapolis, IN, USA（印第安纳大学医学院生物统计学与健康数据科学系）；Department of Computer Science, University of Central Florida, FL, USA（佛罗里达州立大学计算机科学系）
A New Perspective to Boost Performance Fairness for Medical Federated Learning
Australian National University（澳大利亚国立大学）；The Hong Kong University of Science and Technology (Guangzhou)（香港科技大学（广州））；Guangxi Medical University（广西医科大学）；Institute of High Performance Computing (IHPC), Agency for Science, Technology and Research (A*STAR)（新加坡科技研究局高性能计算研究所）；Mohamed bin Zayed University of Artificial Intelligence (MBZUAI)（穆罕默德·本·扎耶德人工智能大学）；The Hong Kong University of Science and Technology（香港科技大学）
Beyond Static Knowledge Messengers: Towards Adaptive, Fair, and Scalable Federated Learning for Medical AI
Department of Computer Science and Software Engineering, Auburn University（计算机科学与软件工程系，阿伯丁大学）；Department of Information and Communication Technology, Bangladesh University of Professionals（信息与通信技术系，孟加拉国专业大学）；Mymensingh Medical College and Hospital（迈明辛医疗学院和医院）
The Chinese University of Hong Kong（香港中文大学）；NVIDIA（英伟达）
How to ensure fairness is an important topic in federated learning (FL). Recent studies have investigated how to reward clients based on their contribution (collaboration fairness), and how to achieve uniformity of performance across clients (performance fairness). Despite achieving progress on either one, we argue that it is critical to consider them together, in order to engage and motivate more diverse clients joining FL to derive a high-quality global model. In this work, we propose a novel method to optimize both types of fairness simultaneously. Specifically, we propose to estimate clien…
University of Maryland, College Park（马里兰大学帕克分校）；Max Planck Institute for Software Systems（马克斯·普朗克软件系统研究所）
As machine learning algorithms have been widely deployed across applications, many concerns have been raised over the fairness of their predictions, especially in high stakes settings (such as facial recognition and medical imaging). To respond to these concerns, the community has proposed and formalized various notions of fairness as well as methods for rectifying unfair behavior. While fairness constraints have been studied extensively for classical models, the effectiveness of methods for imposing fairness on deep neural networks is unclear. In this paper, we observe that these large models…
University of Illinois at Urbana-Champaign（伊利诺伊大学厄巴纳-香槟分校）
With the increasing application of machine learning in high-stake decision-making problems, potential algorithmic bias towards people from certain social groups poses negative impacts on individuals and our society at large. In the real-world scenario, many such problems involve positive and unlabeled data such as medical diagnosis, criminal risk assessment and recommender systems. For instance, in medical diagnosis, only the diagnosed diseases will be recorded (positive) while others will not (unlabeled). Despite the large amount of existing work on fairness-aware machine learning in the (sem…
MultiFair: Multimodal Balanced Fairness-Aware Medical Classification with Dual-Level Gradient Modulation
School of Computing and Informatics, University of Louisiana at Lafayette（路易斯安那州立大学拉法叶分校计算机与信息学学院）；Louisiana Center for Health Innovation and College of Nursing & Health Sciences, University of Louisiana at Lafayette（路易斯安那州立大学拉法叶分校健康创新中心及护理与健康科学学院）；Massachusetts Eye and Ear, Harvard Medical School（哈佛医学院马萨诸塞眼耳医院）；Department of Computer Science, University of Central Florida（佛罗里达州立大学计算机科学系）；Department of Electrical Engineering and Computer Science, Florida Atlantic University（佛罗里达Atlantic大学电子工程与计算机科学系）
University of Cambridge（剑桥大学）；MRC Biostatistics Unit, University of Cambridge（剑桥大学医学研究委员会生物统计组）；Sapienza University of Rome（罗马第一大学）；The Alan Turing Institute（阿兰·图灵研究所）；Technical University of Munich（慕尼黑工业大学）；Helmholtz Munich（亥姆霍兹慕尼黑中心）
While interest in the application of machine learning to improve healthcare has grown tremendously in recent years, a number of barriers prevent deployment in medical practice. A notable concern is the potential to exacerbate entrenched biases and existing health disparities in society. The area of fairness in machine learning seeks to address these issues of equity; however, appropriate approaches are context-dependent, necessitating domain-specific consideration. We focus on clinical trials, i.e., research studies conducted on humans to evaluate medical treatments. Clinical trials are a rela…
Indian Institute of Science（印度科学学院）；Microsoft Research（微软研究院）
In this paper, we study the problem of fair classification in the presence of prior probability shifts, where the training set distribution differs from the test set. This phenomenon can be observed in the yearly records of several real-world datasets, such as recidivism records and medical expenditure surveys. If unaccounted for, such shifts can cause the predictions of a classifier to become unfair towards specific population subgroups. While the fairness notion called Proportional Equality (PE) accounts for such shifts, a procedure to ensure PE-fairness was unknown. In this work, we propose…
Fair Federated Medical Image Classification Against Quality Shift via Inter-Client Progressive State Matching
School of Electronic Information and Communications, Huazhong University of Science and Technology（华中科技大学电子信息与通信学院）；Lassonde School of Engineering, York University（约克大学拉萨onde工程学院）
Differential privacy for medical deep learning: methods, tradeoffs, and deployment implications
Pattern Recognition Lab, Friedrich-Alexander-Universität Erlangen-Nürnberg（埃朗根-纽伦堡弗里德里希-亚历山大大学模式识别实验室）；Department of Diagnostic and Interventional Radiology, University Hospital RWTH Aachen（亚琛大学诊断与介入放射科）；Department of Radiology, Stanford University（斯坦福大学放射科）；Department of Urology, Stanford University（斯坦福大学泌尿科）
Stamford University（斯坦福德大学）；Charles Sturt University（查尔斯斯特大学）；University of Chittagong（吉大港大学）
Due to the limited availability and high cost of the reverse transcription-polymerase chain reaction (RT-PCR) test, many studies have proposed machine learning techniques for detecting COVID-19 from medical imaging. The purpose of this study is to systematically review, assess, and synthesize research articles that have used different machine learning techniques to detect and diagnose COVID-19 from chest X-ray and CT scan images. A structured literature search was conducted in the relevant bibliographic databases to ensure that the survey solely centered on reproducible and high-quality resear…
UIUC（伊利诺伊大学厄巴纳-香槟分校）；ETH Zurich（苏黎世联邦理工学院）；UC Santa Cruz（加州大学圣克鲁兹分校）
Extensive efforts have been made to understand and improve the fairness of machine learning models based on observational metrics, especially in high-stakes domains such as medical insurance, education, and hiring decisions. However, there is a lack of certified fairness considering the end-to-end performance of an ML model. In this paper, we first formulate the certified fairness of an ML model trained on a given data distribution as an optimization problem based on the model performance loss bound on a fairness constrained distribution, which is within bounded distributional distance with th…
FairDiffusion: Enhancing Equity in Latent Diffusion Models via Fair Bayesian Perturbation
Harvard Medical School（哈佛医学院）；New York University（纽约大学）；New York University Abu Dhabi（纽约大学阿布扎比分校）；Harvard University（哈佛大学）；Broad Institute of MIT and Harvard（麻省理工学院与哈佛大学博德研究所）
This paper evaluates synthetically generated healthcare data for biases and investigates the effect of fairness mitigation techniques on utility-fairness. Privacy laws limit access to health data such as Electronic Medical Records (EMRs) to preserve patient privacy. Albeit essential, these laws hinder research reproducibility. Synthetic data is a viable solution that can enable access to data similar to real healthcare data without privacy risks. Healthcare datasets may have biases in which certain protected groups might experience worse outcomes than others. With the real data having biases, …
University of Cambridge（剑桥大学）；Binghamton University（宾汉姆顿大学）；Beijing University of Posts and Telecommunications（北京邮电大学）
As Facial Expression Recognition (FER) systems become integrated into our daily lives, these systems need to prioritise making fair decisions instead of aiming at higher individual accuracy scores. Ranging from surveillance systems to diagnosing mental and emotional health conditions of individuals, these systems need to balance the accuracy vs fairness trade-off to make decisions that do not unjustly discriminate against specific under-represented demographic groups. Identifying bias as a critical problem in facial analysis systems, different methods have been proposed that aim to mitigate bi…
