Mitigating Biases in Surgical Operating Rooms with Geometry
Computer Aided Medical Procedures, TU Munich, Germany（图宾根大学医学辅助技术部）；Munich Center for Machine Learning, Germany（慕尼黑机器学习中心）；UCL Hawkes Institute, Dept. Computer Science, University College London（伦敦大学学院霍克斯研究所）；Minimally Invasive Surgery, University Hospital of Munich (LMU), Germany（慕尼黑大学医院微创外科）
Tony Danjun Wang, Tobias Czempiel, Nassir Navab, Lennart Bastian
备注：
Extended Abstract, presented at the MICCAI'25 workshop on Collaborative Intelligence and Autonomy in Image-guided Surgery
Deep neural networks are prone to learning spurious correlations, exploiting dataset-specific artifacts rather than meaningful features for prediction. In surgical operating rooms (OR), these manifest through the standardization of smocks and gowns that obscure robust identifying landmarks, introducing model bias for tasks related to modeling OR personnel. Through gradient-based saliency analysis on two public OR datasets, we reveal that CNN models succumb to such shortcuts, fixating on incidental visual cues such as footwear beneath surgical gowns, distinctive eyewear, or other role-specific identifiers. Avoiding such biases is essential for the next generation of intelligent assistance systems in the OR, which should accurately recognize personalized workflow traits, such as surgical skill level or coordination with other staff members. We address this problem by encoding personnel as 3D point cloud sequences, disentangling identity-relevant shape and motion patterns from appearance-based confounders. Our experiments demonstrate that while RGB and geometric methods achieve comparable performance on datasets with apparent simulation artifacts, RGB models suffer a 12% accuracy drop in realistic clinical settings with decreased visual diversity due to standardizations. This performance gap confirms that geometric representations capture more meaningful biometric features, providing an avenue to developing robust methods of modeling humans in the OR.
Machine learning models are known to perpetuate and even amplify the biases present in the data. However, these data biases frequently do not become apparent until after the models are deployed. Our work tackles this issue and enables the preemptive analysis of large-scale datasets. REVISE (REvealing VIsual biaSEs) is a tool that assists in the investigation of a visual dataset, surfacing potential biases along three dimensions: (1) object-based, (2) person-based, and (3) geography-based. Object-based biases relate to the size, context, or diversity of the depicted objects. Person-based metric…
MeDi: Metadata-Guided Diffusion Models for Mitigating Biases in Tumor Classification
Machine Learning Group, Technische Universität Berlin, Germany The Konrad Zuse School of Excellence in Learning；Intelligent Systems (ELIZA), Germany BIFOLD – Berlin Institute for the Foundations of Learning
Quantifying and Mitigating Unimodal Biases in Multimodal Large Language Models: A Causal Perspective
Shanghai AI Laboratory（上海人工智能实验室）；State Key Laboratory of General Artificial Intelligence, Peking University（北京大学通用人工智能国家重点实验室）；School of Intelligence Science and Technology, Peking University（北京大学智能科学与技术学院）；School of Computer Science, Fudan University（复旦大学计算机科学技术学院）；Shanghai Artificial Intelligence Laboratory（上海人工智能实验室）
University of Virginia（弗吉尼亚大学）；University of California Los Angeles（加州大学洛杉矶分校）；Allen Institute for Artificial Intelligence（艾伦人工智能研究所）
In this work, we present a framework to measure and mitigate intrinsic biases with respect to protected variables --such as gender-- in visual recognition tasks. We show that trained models significantly amplify the association of target labels with gender beyond what one would expect from biased datasets. Surprisingly, we show that even when datasets are balanced such that each label co-occurs equally with each gender, learned models amplify the association between labels and gender, as much as if data had not been balanced! To mitigate this, we adopt an adversarial approach to remove unwante…
The performance of a computer vision model depends on the size and quality of its training data. Recent studies have unveiled previously-unknown composition biases in common image datasets which then lead to skewed model outputs, and have proposed methods to mitigate these biases. However, most existing works assume that human-generated annotations can be considered gold-standard and unbiased. In this paper, we reveal that this assumption can be problematic, and that special care should be taken to prevent models from learning such annotation biases. We focus on facial expression recognition a…
University of Rochester（罗切斯特大学）；Kitware, Inc.（Kitware公司）
Deep image classifiers have been found to learn biases from datasets. To mitigate the biases, most previous methods require labels of protected attributes (e.g., age, skin tone) as full-supervision, which has two limitations: 1) it is infeasible when the labels are unavailable; 2) they are incapable of mitigating unknown biases -- biases that humans do not preconceive. To resolve those problems, we propose Debiasing Alternate Networks (DebiAN), which comprises two networks -- a Discoverer and a Classifier. By training in an alternate manner, the discoverer tries to find multiple unknown biases…
Exposing and Mitigating Calibration Biases and Demographic Unfairness in MLLM Few-Shot In-Context Learning for Medical Image Classification
Centre for Intelligent Machines, McGill University, Montreal, Canada（智能机器中心，麦吉尔大学，加拿大）；Mila -- Quebec AI Institute, Montreal, Canada（魁北克人工智能研究所）；Stanford University, Stanford, USA（斯坦福大学）；University of Copenhagen, Copenhagen, Denmark（哥本哈根大学）
Pontifícia Universidade Católica do Rio Grande do Sul（南里奥格兰德天主教大学）；Universidade Federal de Minas Gerais（米纳斯吉拉斯联邦大学）
Facial analysis models are increasingly applied in real-world applications that have significant impact on peoples' lives. However, as literature has shown, models that automatically classify facial attributes might exhibit algorithmic discrimination behavior with respect to protected groups, potentially posing negative impacts on individuals and society. It is therefore critical to develop techniques that can mitigate unintended biases in facial classifiers. Hence, in this work, we introduce a novel learning method that combines both subjective human-based labels and objective annotations bas…
Reliably detecting diseases using relevant biological information is crucial for real-world applicability of deep learning techniques in medical imaging. We debias deep learning models during training against unknown bias - without preprocessing/filtering the input beforehand or assuming specific knowledge about its distribution or precise nature in the dataset. We use control regions as surrogates that carry information regarding the bias, employ the classifier model to extract features, and suppress biased intermediate features with our custom, modular DecorreLayer. We evaluate our method on…
Bogazici University（博阿齐奇大学）；University of Cambridge（剑桥大学）；Beijing University of Posts and Telecommunications（北京邮电大学）
As affective robots become integral in human life, these agents must be able to fairly evaluate human affective expressions without discriminating against specific demographic groups. Identifying bias in Machine Learning (ML) systems as a critical problem, different approaches have been proposed to mitigate such biases in the models both at data and algorithmic levels. In this work, we propose Continual Learning (CL) as an effective strategy to enhance fairness in Facial Expression Recognition (FER) systems, guarding against biases arising from imbalances in data distributions. We compare diff…
