In healthcare, predictive models increasingly inform patient-level decisions, yet little attention is paid to the variability in individual risk estimates and its impact on treatment decisions. For overparameterized models, now standard in machine learning, a substantial source of variability often goes undetected. Even when the data and model architecture are held fixed, randomness introduced by optimization and initialization can lead to materially different risk estimates for the same patient. This problem is largely obscured by standard evaluation practices, which rely on aggregate performance metrics (e.g., log-loss, accuracy) that are agnostic to individual-level stability. As a result, models with indistinguishable aggregate performance can nonetheless exhibit substantial procedural arbitrariness, which can undermine clinical trust. We propose an evaluation framework that quantifies individual-level prediction instability by using two complementary diagnostics: empirical prediction interval width (ePIW), which captures variability in continuous risk estimates, and empirical decision flip rate (eDFR), which measures instability in threshold-based clinical decisions. We apply these diagnostics to simulated data and GUSTO-I clinical dataset. Across observed settings, we find that for flexible machine-learning models, randomness arising solely from optimization and initialization can induce individual-level variability comparable to that produced by resampling the entire training dataset. Neural networks exhibit substantially greater instability in individual risk predictions compared to logistic regression models. Risk estimate instability near clinically relevant decision thresholds can alter treatment recommendations. These findings that stability diagnostics should be incorporated into routine model validation for assessing clinical reliability.
Clinical Recommender System: Predicting Medical Specialty Diagnostic Choices with Neural Network Ensembles
Stanford Center for Biomedical Informatics Research, Stanford University(斯坦福大学生物医学信息学研究中心);Division of Endocrinology, Stanford University School of Medicine(斯坦福大学医学院内分泌科);Division of Hospital Medicine, Stanford University(斯坦福大学医院医学部)
Automated Ensemble Multimodal Machine Learning for Healthcare
Department of Statistics, University of Oxford(统计系,牛津大学);Division of Medical Image Computing, German Cancer Research Center (DKFZ)(医学影像计算部,德国癌症研究中心(DKFZ));Medical Faculty Heidelberg, Heidelberg University(海德堡大学医学系);École Polytechnique Fédérale de Lausanne(洛桑联邦理工学院);Pattern Analysis and Learning Group, Department of Radiation Oncology, Heidelberg University Hospital(模式分析与学习小组,海德堡大学医院放射肿瘤学系);National Center for Tumor Diseases (NCT) Heidelberg(海德堡国家肿瘤疾病中心(NCT));Department of Applied Mathematics and Theoretical Physics, University of Cambridge(应用数学与理论物理系,剑桥大学)
Adversarial Fragility and Language Vulnerability in Clinical AI: A Systematic Audit of Diagnostic Collapse Under Imperceptible Perturbations and Cross-Lingual Drift in Low-Resource Healthcare Settings
Centre for Clinical Intelligence & Safety(临床智能与安全中心);Tomorrow University of Applied Sciences(明天应用科学大学)
Large Language Models Illuminate a Progressive Pathway to Artificial Healthcare Assistant: A Review
Peking University(北京大学);Peking University Cancer Hospital and Institute(北京大学肿瘤医院暨北京市肿瘤防治研究所);National Engineering Laboratory for Big Data Analysis and Applications, Peking University(北京大学大数据分析与应用技术国家工程实验室);Peking University Changsha Institute for Computing and Digital Economy(北京大学长沙计算与数字经济研究院)
Designing a Deep Learning-Driven Resource-Efficient Diagnostic System for Metastatic Breast Cancer: Reducing Long Delays of Clinical Diagnosis and Improving Patient Survival in Developing Countries
University of Maryland, Baltimore County(马里兰大学巴尔的摩县分校);Meta Platforms Inc.(元平台公司)
Beyond One-Time Validation: A Framework for Adaptive Validation of Prognostic and Diagnostic AI-based Medical Devices
x-cardiac GmbH(x-cardiac有限公司);University of Tübingen(图宾根大学);Deutsches Herzzentrum der Charité(夏里特德国心脏中心);German Centre for Cardiovascular Research(德国心血管研究中心);Berlin Institute for the Foundations of Learning and Data(柏林学习与数据基础研究所)
Debias-CLR: A Contrastive Learning Based Debiasing Method for Algorithmic Fairness in Healthcare Applications
Wright State University(莱特州立大学);Cincinnati Children’s Hospital Medical Center(辛辛那提儿童医院医疗中心);Kairos Research LLC(Kairos Research有限责任公司);University of Illinois Chicago(伊利诺伊大学芝加哥分校)
Retrieval-Augmented Interpretable Learning: Towards Task-Specific Zero-Shot Models in Healthcare
Carnegie Mellon University(卡内基梅隆大学);University of Wisconsin–Madison(威斯康星大学麦迪逊分校);Mohamed bin Zayed University of AI(穆罕默德·本·扎耶德人工智能大学);GenBio AI(基因生物人工智能公司);Intel(英特尔公司)
Easydiagnos: a framework for accurate feature selection for automatic diagnosis in smart healthcare
Dr. B. C. Roy Engineering College(B.C.罗伊工程学院);Bengal College of Engg. and Tech.(孟加拉工程技术学院);National Institute of Technology(印度国家理工学院);University of North Texas(北得克萨斯大学)
本文提出 Easydiagnos 框架,通过融合遗传算法、可解释 AI 和排列组合技术的 AFE 算法优化医疗特征选择,并在多个数据集上验证其可提升临床决策的准确性与可解释性。
Real-Time Magnetic Tracking and Diagnosis of COVID-19 via Machine Learning
University of South Florida(南佛罗里达大学);Hue Central Hospital(顺化中央医院);Riga Stradinš University(里加斯特拉迪宁什大学);Vietnam National University(越南国家大学);University Medical Center(大学医学中心);Traditional Medicine Hospital, Ministry of Public Security(公安部传统医学医院)
Signal Fidelity Index-Aware Calibration for Dementia Predictions Across Heterogeneous Real-World Data
Department of Medicine, Massachusetts General Hospital(麻省总医院内科部);Department of Biostatistics, Harvard T.H. Chan School of Public Health(哈佛T.H. Chan公共卫生学院生物统计学部);Department of Medicine, Brigham and Women’s Hospital(布里洛妇产科医院内科部)
Reconstruction of Patient-Specific Confounders in AI-based Radiologic Image Interpretation using Generative Pretraining
Else Kroener Fresenius Center for Digital Health (EKFZ)(埃尔泽·克勒纳·菲森纽斯数字健康中心(EKFZ));Technical University Dresden(德累斯顿工业大学);University Hospital Aachen(亚琛大学医院);UC Berkeley(加州大学伯克利分校);University Hospital Zurich(苏黎世大学医院);University of Zurich(苏黎世大学);Center for Artificial Intelligence in Medicine and Imaging (AIMI)(医学与成像人工智能中心(AIMI));Stanford University(斯坦福大学);University Hospital Dresden(德累斯顿大学医院);National Center for Tumor Diseases (NCT)(国家肿瘤疾病中心(NCT))
A Foundational Framework and Methodology for Personalized Early and Timely Diagnosis
University of Cambridge(剑桥大学);Heidelberg University(海德堡大学);University of Liverpool(利物浦大学);Roche Innovation Center(罗氏创新中心);The Cambridge Centre for AI in Medicine(剑桥医学人工智能中心);University of California Los Angeles(加州大学洛杉矶分校)