An AI-Guided Data Centric Strategy to Detect and Mitigate Biases in Healthcare Datasets
Icahn School of Medicine at Mount Sinai（西奈山伊坎医学院）；University of Michigan Medical School（密歇根大学医学院）；Beth Israel Deaconess Medical Center / Harvard Medical School（贝斯以色列女执事医疗中心/哈佛医学院）；IBM Research（IBM研究院）
Faris F. Gulamali, Ashwin S. Sawant, Lora Liharska, Carol R. Horowitz, Lili Chan, Patricia H. Kovatch, Ira Hofer, Karandeep Singh, Lynne D. Richardson, Emmanuel… 展开作者
Faris F. Gulamali, Ashwin S. Sawant, Lora Liharska, Carol R. Horowitz, Lili Chan, Patricia H. Kovatch, Ira Hofer, Karandeep Singh, Lynne D. Richardson, Emmanuel Mensah, Alexander W Charney, David L. Reich, Jianying Hu, Girish N. Nadkarni
The adoption of diagnosis and prognostic algorithms in healthcare has led to concerns about the perpetuation of bias against disadvantaged groups of individuals. Deep learning methods to detect and mitigate bias have revolved around modifying models, optimization strategies, and threshold calibration with varying levels of success. Here, we generate a data-centric, model-agnostic, task-agnostic approach to evaluate dataset bias by investigating the relationship between how easily different groups are learned at small sample sizes (AEquity). We then apply a systematic analysis of AEq values across subpopulations to identify and mitigate manifestations of racial bias in two known cases in healthcare - Chest X-rays diagnosis with deep convolutional neural networks and healthcare utilization prediction with multivariate logistic regression. AEq is a novel and broadly applicable metric that can be applied to advance equity by diagnosing and remediating bias in healthcare datasets.
Benchmarking Instance-Centric Counterfactual Algorithms for XAI: From White Box to Black Box
Human Technology Institute, University of Technology Sydney（悉尼科技大学人类技术研究所）；School of Information Systems, Queensland University of Technology（昆士兰科技大学信息系统学院）；INESC-ID Lisboa / Instituto Superior Técnico, ULisboa（INESC-ID里斯本/里斯本大学高等技术学院）
As machine learning methods are deployed in real-world settings such as healthcare, legal systems, and social science, it is crucial to recognize how they shape social biases and stereotypes in these sensitive decision-making processes. Among such real-world deployments are large-scale pretrained language models (LMs) that can be potentially dangerous in manifesting undesirable representational biases - harmful biases resulting from stereotyping that propagate negative generalizations involving gender, race, religion, and other social constructs. As a step towards improving the fairness of LMs…
Entity alignment is to find identical entities in different knowledge graphs (KGs) that refer to the same real-world object. Embedding-based entity alignment techniques have been drawing a lot of attention recently because they can help solve the issue of symbolic heterogeneity in different KGs. However, in this paper, we show that the progress made in the past was due to biased and unchallenging evaluation. We highlight two major flaws in existing datasets that favor embedding-based entity alignment techniques, i.e., the isomorphic graph structures in relation triples and the weak heterogenei…
Person-Centric Annotations of LAION-400M: Auditing Bias and Its Transfer to Models
Technical University of Munich, Munich Center for Machine Learning (MCML), MDSI（慕尼黑工业大学，慕尼黑机器学习中心（MCML），MDSI）；LTCI, Télécom Paris, Institut Polytechnique de Paris, France（法国巴黎理工学院，巴黎电信学院，LTCI）；Helmholtz Munich（亥姆霍兹慕尼黑中心）；University of California, Berkeley（加州大学伯克利分校）
Korea University（高丽大学）；Upstage；Technical University of Darmstadt（达姆施塔特工业大学）
Leaderboard systems allow researchers to objectively evaluate Natural Language Processing (NLP) models and are typically used to identify models that exhibit superior performance on a given task in a predetermined setting. However, we argue that evaluation on a given test dataset is just one of many performance indications of the model. In this paper, we claim leaderboard competitions should also aim to identify models that exhibit the best performance in a real-world setting. We highlight three issues with current leaderboard systems: (1) the use of a single, static test set, (2) discrepancy …
Reinforcement learning (RL) has been extensively researched for enhancing human-environment interactions in various human-centric tasks, including e-learning and healthcare. Since deploying and evaluating policies online are high-stakes in such tasks, off-policy evaluation (OPE) is crucial for inducing effective policies. In human-centric environments, however, OPE is challenging because the underlying state is often unobservable, while only aggregate rewards can be observed (students' test scores or whether a patient is released from the hospital eventually). In this work, we propose a human-…
Explainable artificial intelligence is increasingly used in machine learning (ML) based decision-making systems in healthcare. However, little research has compared the utility of different explanation methods in guiding healthcare experts for patient care. Moreover, it is unclear how useful, understandable, actionable and trustworthy these methods are for healthcare experts, as they often require technical ML knowledge. This paper presents an explanation dashboard that predicts the risk of diabetes onset and explains those predictions with data-centric, feature-importance, and example-based e…
Blockchain-empowered Federated Learning for Healthcare Metaverses: User-centric Incentive Mechanism with Optimal Data Freshness
Guangdong University of Technology（广东工业大学）；Singapore University of Technology and Design（新加坡科技设计大学）；Nanyang Technological University（南洋理工大学）；Nanjing University of Aeronautics and Astronautics（南京航空航天大学）
Beihang University（北京航空航天大学）；Sun Yat-sen University（中山大学）；Pazhou Lab（琶洲实验室）；ByteDance AI Lab（字节跳动人工智能实验室）；University of Sydney（悉尼大学）
In this work, we introduce a novel task - Humancentric Spatio-Temporal Video Grounding (HC-STVG). Unlike the existing referring expression tasks in images or videos, by focusing on humans, HC-STVG aims to localize a spatiotemporal tube of the target person from an untrimmed video based on a given textural description. This task is useful, especially for healthcare and security-related applications, where the surveillance videos can be extremely long but only a specific person during a specific period of time is concerned. HC-STVG is a video grounding task that requires both spatial (where) and…
This paper evaluates synthetically generated healthcare data for biases and investigates the effect of fairness mitigation techniques on utility-fairness. Privacy laws limit access to health data such as Electronic Medical Records (EMRs) to preserve patient privacy. Albeit essential, these laws hinder research reproducibility. Synthetic data is a viable solution that can enable access to data similar to real healthcare data without privacy risks. Healthcare datasets may have biases in which certain protected groups might experience worse outcomes than others. With the real data having biases, …
Robust Multimodal Representation Learning in Healthcare
University of California, Davis（加州大学戴维斯分校）；HKUST (GZ)（香港科技大学）；University of Rochester（罗切斯特大学）；Tongji University（同济大学）；Georgia Institute of Technology（佐治亚理工学院）；Fudan University（复旦大学）；The University of British Columbia（不列颠哥伦比亚大学）
University of Amsterdam（阿姆斯特丹大学）；The Netherlands Cancer Institute（荷兰癌症研究所）；Technical University of Denmark（丹麦技术大学）；University of Copenhagen（哥本哈根大学）；Copenhagen University Hospital（哥本哈根大学医院）；Max Planck Institute for Intelligent Systems（马克斯·普朗克智能系统研究所）
Understanding which inductive biases could be helpful for the unsupervised learning of object-centric representations of natural scenes is challenging. In this paper, we systematically investigate the performance of two models on datasets where neural style transfer was used to obtain objects with complex textures while still retaining ground-truth annotations. We find that by using a single module to reconstruct both the shape and visual appearance of each object, the model learns more useful representations and achieves better object separation. In addition, we observe that adjusting the lat…
A range of methods with suitable inductive biases exist to learn interpretable object-centric representations of images without supervision. However, these are largely restricted to visually simple images; robust object discovery in real-world sensory datasets remains elusive. To increase the understanding of such inductive biases, we empirically investigate the role of "reconstruction bottlenecks" for scene decomposition in GENESIS, a recent VAE-based model. We show such bottlenecks determine reconstruction and segmentation quality and critically influence model behaviour.
Object-Centric Temporal Consistency via Conditional Autoregressive Inductive Biases
Delft University of Technology（代尔夫特理工大学）；The University of Tokyo（东京大学）；Mila, University of Montreal（蒙特利尔大学米拉研究所）；Google DeepMind（谷歌DeepMind）；Agency for Science, Technology and Research（新加坡科技研究局）；Nanyang Technological University（南洋理工大学）
Predictive models are increasingly used to make various consequential decisions in high-stakes domains such as healthcare, finance, and policy. It becomes critical to ensure that these models make accurate predictions, are robust to shifts in the data, do not rely on spurious features, and do not unduly discriminate against minority groups. To this end, several approaches spanning various areas such as explainability, fairness, and robustness have been proposed in recent literature. Such approaches need to be human-centered as they cater to the understanding of the models to their users. Howev…
Learning Decomposable and Debiased Representations via Attribute-Centric Information Bottlenecks
School of Computing and Augmented Intelligence（计算与增强智能学院）；Arizona State University（亚利桑那州立大学）；School of Arts, Media and Engineering（艺术、媒体与工程学院）；School of Life Sciences（生命科学学院）
提出去偏框架 Debiasing Global Workspace，利用基于注意力的信息瓶颈学习可分解的内在与偏置属性组合表示，提升模型分布外泛化能力并区分内在与偏置特征。
Providing front-line health workers in low- and middle- income countries with recommendations and predictions to improve health outcomes can have a tremendous impact on reducing healthcare inequalities, for instance by helping to prevent the thousands of maternal and newborn deaths that occur every day. To that end, we are developing a data-centric machine learning platform that leverages the behavioral logs from a wide range of mobile health applications running in those countries. Here we describe the platform architecture, focusing on the details that help us to maximize the quality and org…
Towards deployment-centric multimodal AI beyond vision and language
Centre for Machine Intelligence, University of Sheffield, Sheffield, UK（机器智能中心，谢菲尔德大学）；School of Computer Science, University of Sheffield, Sheffield, UK（计算机科学学院，谢菲尔德大学）；The Alan Turing Institute, London, UK（艾伦·图灵研究所，伦敦）；Department of Metabolism, Digestion and Reproduction, Imperial College London, London, UK（代谢、消化与生殖部门，伦敦帝国学院）；Department of Applied AI, Simula Research Laboratory, Oslo, Norway（应用人工智能部门，Simula研究实验室，奥斯陆）；Information School, University of Sheffield, Sheffield, UK（信息学院，谢菲尔德大学）；NHS England, Leeds, UK（英国英格兰国家医疗服务体系，利兹）；Institute of Health Informatics, University College London, London, UK（健康信息研究所，伦敦大学学院）；Department of Engineering, King’s College London, London, UK（工程部门，伦敦国王学院）；Department of Computing Science, University of Aberdeen, Aberdeen, UK（计算科学部门，阿伯丁大学）；School of Informatics, University of Edinburgh, Edinburgh, UK（信息学院，爱丁堡大学）；School of Engineering Mathematics and Technology, University of Bristol, Bristol, UK（工程数学与技术学院，布里斯托尔大学）；Department of Informatics, King’s College London, London, UK（信息部门，伦敦国王学院）；Department of Earth Sciences, University of Cambridge, Cambridge, UK（地球科学部门，剑桥大学）；Department of Computer Science, University of Manchester, Manchester, UK（计算机科学部门，曼彻斯特大学）
Towards Data-Centric AI: A Comprehensive Survey of Traditional, Reinforcement, and Generative Approaches for Tabular Data Transformation
University of Kansas（堪萨斯大学）；Southwestern University of Finance and Economics（西南财经大学）；Arizona State University（亚利桑那州立大学）；Portland State University（波特兰州立大学）；Computer Network Information Center, Chinese Academy of Sciences（中国科学院计算机网络信息中心）；University of Macau（澳门大学）；The Hong Kong University of Science and Technology（香港科技大学）
Courant Institute of Mathematical Sciences（科朗数学科学研究所）；Center for Data Science（数据科学中心）；New York University（纽约大学）；New York University Langone Medical Center（纽约大学兰贡医学中心）
As machine learning (ML) models, trained on real-world datasets, become common practice, it is critical to measure and quantify their potential biases. In this paper, we focus on renal failure and compare a commonly used traditional risk score, Tangri, with a more powerful machine learning model, which has access to a larger variable set and trained on 1.6 million patients' EHR data. We will compare and discuss the generalization and applicability of these two models, in an attempt to quantify biases of status quo clinical practice, compared to ML-driven models.
Memorization Control in Diffusion Models from Denoising-centric Perspective
Department of Computer Science（计算机科学系）；Brunel University of London（布鲁内尔大学）；Hanoi University of Science and Technology（河内科学技术大学）；Faculty of Electrical and Electronic Engineering School of Engineering（电气电子工程系工程学院）；Phenikaa University（Phenikaa大学）；Greenwich Vietnam FPT University（格林威治越南FPT大学）；College of Engineering（工程学院）；Shibaura Institute of Technology（Shibaura技术学院）
Scuola Normale Superiore（比萨高等师范学院）；ISI Foundation（ISI基金会）；University of Pisa（比萨大学）
The pervasive application of algorithmic decision-making is raising concerns on the risk of unintended bias in AI systems deployed in critical settings such as healthcare. The detection and mitigation of biased models is a very delicate task which should be tackled with care and involving domain experts in the loop. In this paper we introduce FairLens, a methodology for discovering and explaining biases. We show how our tool can be used to audit a fictional commercial black-box model acting as a clinical decision support system. In this scenario, the healthcare facility experts can use FairLen…
University of California, Irvine（加州大学欧文分校）；University of Turku（图尔库大学）
Smart eHealth applications deliver personalized and preventive digital healthcare services to clients through remote sensing, continuous monitoring, and data analytics. Smart eHealth applications sense input data from multiple modalities, transmit the data to edge and/or cloud nodes, and process the data with compute intensive machine learning (ML) algorithms. Run-time variations with continuous stream of noisy input data, unreliable network connection, computational requirements of ML algorithms, and choice of compute placement among sensor-edge-cloud layers affect the efficiency of ML-driven…
Clustering is one of the most universal approaches for understanding complex data. A pivotal aspect of clustering analysis is quantitatively comparing clusterings; clustering comparison is the basis for many tasks such as clustering evaluation, consensus clustering, and tracking the temporal evolution of clusters. In particular, the extrinsic evaluation of clustering methods requires comparing the uncovered clusterings to planted clusterings or known metadata. Yet, as we demonstrate, existing clustering comparison measures have critical biases which undermine their usefulness, and no measure a…
University of California, Los Angeles（加利福尼亚大学洛杉矶分校）
Auditing machine learning-based (ML) healthcare tools for bias is critical to preventing patient harm, especially in communities that disproportionately face health inequities. General frameworks are becoming increasingly available to measure ML fairness gaps between groups. However, ML for health (ML4H) auditing principles call for a contextual, patient-centered approach to model assessment. Therefore, ML auditing tools must be (1) better aligned with ML4H auditing principles and (2) able to illuminate and characterize communities vulnerable to the most harm. To address this gap, we propose s…
Generative AI-Driven Human Digital Twin in IoT-Healthcare: A Comprehensive Survey
College of Computer Science and Technology, Nanjing University of Aeronautics and Astronautics（南京航空航天大学计算机科学与技术学院）；School of Computer Science and Engineering, Nanyang Technological University（南洋理工大学计算机科学与工程学院）；School of Automation, Guangdong University of Technology（广东工业大学自动化学院）
Object-centric representations are a promising path toward more systematic generalization by providing flexible abstractions upon which compositional world models can be built. Recent work on simple 2D and 3D datasets has shown that models with object-centric inductive biases can learn to segment and represent meaningful objects from the statistical structure of the data alone without the need for any supervision. However, such fully-unsupervised methods still fail to scale to diverse realistic data, despite the use of increasingly complex inductive biases such as priors for the size of object…
Missing values exist in nearly all clinical studies because data for a variable or question are not collected or not available. Inadequate handling of missing values can lead to biased results and loss of statistical power in analysis. Existing models usually do not consider privacy concerns or do not utilise the inherent correlations across multiple features to impute the missing values. In healthcare applications, we are usually confronted with high dimensional and sometimes small sample size datasets that need more effective augmentation or imputation techniques. Besides, imputation and aug…
BhashaSetu: A Data-Centric Approach to Low-Resource Machine Translation
Department of Computer Engineering and Information Technology, Veermata Jijabai Technological Institute, Mumbai（孟买韦尔马塔·吉贾拜技术学院计算机工程与信息技术系）；Tübingen AI Center, University of Tübingen, Germany（图宾根大学图宾根人工智能中心，德国）
