Trustworthy clinical AI solutions: a unified review of uncertainty quantification in deep learning models for medical image analysis
Univ. Grenoble Alpes（格勒诺布尔阿尔卑斯大学）；Inserm（法国国家健康与医学研究院）；Pixyl Research and Development Laboratory（Pixyl研发实验室）；Inria（法国国家信息与自动化研究所）；CNRS（法国国家科学研究中心）；Grenoble INP（格勒诺布尔理工学院）
Benjamin Lambert, Florence Forbes, Alan Tucholka, Senan Doyle, Harmonie Dehaene, Michel Dojat
The full acceptance of Deep Learning (DL) models in the clinical field is rather low with respect to the quantity of high-performing solutions reported in the literature. Particularly, end users are reluctant to rely on the rough predictions of DL models. Uncertainty quantification methods have been proposed in the literature as a potential response to reduce the rough decision provided by the DL black box and thus increase the interpretability and the acceptability of the result by the final user. In this review, we propose an overview of the existing methods to quantify uncertainty associated to DL predictions. We focus on applications to medical image analysis, which present specific challenges due to the high dimensionality of images and their quality variability, as well as constraints associated to real-life clinical routine. We then discuss the evaluation protocols to validate the relevance of uncertainty estimates. Finally, we highlight the open challenges of uncertainty quantification in the medical field.
Despite numerous studies of deep autoencoders (AEs) for unsupervised anomaly detection, AEs still lack a way to express uncertainty in their predictions, crucial for ensuring safe and trustworthy machine learning systems in high-stake applications. Therefore, in this work, the formulation of Bayesian autoencoders (BAEs) is adopted to quantify the total anomaly uncertainty, comprising epistemic and aleatoric uncertainties. To evaluate the quality of uncertainty, we consider the task of classifying anomalies with the additional option of rejecting predictions of high uncertainty. In addition, we…
Towards Trustworthy AI for Glioma Diagnosis: A Task-Aware Evaluation of Uncertainty Quantification
Erasmus MC, University Medical Center Rotterdam（鹿特丹大学医学中心伊拉斯谟MC）；Amsterdam UMC, University of Amsterdam（阿姆斯特丹大学医学中心阿姆斯特丹UMC）；GE HealthCare（GE医疗）；Erasmus MC Cancer Institute（伊拉斯谟MC癌症研究所）；Medical Delta（医疗三角洲）
Trustworthy MRI Reconstruction via Bayesian Uncertainty Quantification with Sparsity Prior Models
Department of Computer Science, University of Exeter（埃克塞特大学计算机科学系）；UCL Hawkes Institute, Department of Computer Science, University College London（伦敦大学学院计算机科学系霍克斯研究所）
Beyond Uncertainty Quantification: Learning Uncertainty for Trust-Informed Neural Network Decisions - A Case Study in COVID-19 Classification
Faculty of Engineering & IT, University of Technology Sydney（技术悉尼大学工程与信息学院）；University Research and Innovation Center (EKIK), Óbuda University（奥布达大学研究与创新中心）
This paper proposes a new data-driven method for the reliable prediction of power system post-fault trajectories. The proposed method is based on the fundamentally new concept of Deep Operator Networks (DeepONets). Compared to traditional neural networks that learn to approximate functions, DeepONets are designed to approximate nonlinear operators. Under this operator framework, we design a DeepONet to (1) take as inputs the fault-on trajectories collected, for example, via simulation or phasor measurement units, and (2) provide as outputs the predicted post-fault trajectories. In addition, we…
Uncertainty quantification by ensemble learning is explored in terms of an application from computational optical form measurements. The application requires to solve a large-scale, nonlinear inverse problem. Ensemble learning is used to extend a recently developed deep learning approach for this application in order to provide an uncertainty quantification of its predicted solution to the inverse problem. By systematically inserting out-of-distribution errors as well as noisy data the reliability of the developed uncertainty quantification is explored. Results are encouraging and the proposed…
CONNECT-AI Research Center, Yonsei University College of Medicine（延世大学医学院CONNECT-AI研究中心）；Gordon Center for Medical Imaging, Massachusetts General Hospital（麻省总医院戈登医学影像中心）；Harvard Medical School（哈佛医学院）
Medical imaging, including MRI, CT, and Ultrasound, plays a vital role in clinical decisions. Accurate segmentation is essential to measure the structure of interest from the image. However, manual segmentation is highly operator-dependent, which leads to high inter and intra-variability of quantitative measurements. In this paper, we explore the feasibility that Bayesian predictive distribution parameterized by deep neural networks can capture the clinicians' inter-intra variability. By exploring and analyzing recently emerged approximate inference schemes, we evaluate whether approximate Bay…
University of Michigan（密歇根大学）；Catalysis Science and Technology Institute, University of Michigan（密歇根大学催化科学与技术研究所）
It is critical that machine learning (ML) model predictions be trustworthy for high-throughput catalyst discovery approaches. Uncertainty quantification (UQ) methods allow estimation of the trustworthiness of an ML model, but these methods have not been well explored in the field of heterogeneous catalysis. Herein, we investigate different UQ methods applied to a crystal graph convolutional neural network (CGCNN) to predict adsorption energies of molecules on alloys from the Open Catalyst 2020 (OC20) dataset, the largest existing heterogeneous catalyst dataset. We apply three UQ methods to the…
FUTURE-AI: Guiding Principles and Consensus Recommendations for Trustworthy Artificial Intelligence in Medical Imaging
Barcelona Artificial Intelligence in Medicine Lab (BCN-AIM)（巴塞罗那医学人工智能实验室（BCN-AIM））；Maggioli SPA（马焦利股份公司）；Fundació TIC Salut i Social（信息与通信技术社会基金会）；Generalitat de Catalunya（加泰罗尼亚政府）；Hospital Universitario y Politécnico La Fe（拉费大学及综合医院）；Health Research Institute（健康研究所）；Foundation for Research and Technology（研究与技术基金会）；National Research Council（国家研究委员会）；Maastricht University（马斯特里赫特大学）
Comprehensive framework for evaluation of deep neural networks in detection and quantification of lymphoma from PET/CT images: clinical insights, pitfalls, and observer agreement analyses
University of British Columbia（英属哥伦比亚大学）；BC Cancer Research Institute（BC癌症研究所）；Microsoft AI for Good Lab（微软AI公益实验室）；Pomeranian Medical University（波美拉尼亚医科大学）；BC Children’s Hospital（BC儿童医院）；St. Mary’s Hospital（圣玛丽医院）；BC Cancer（BC癌症中心）
Metropolis-Scale Resilient and Trustworthy Traffic Flow Inference Using Multi-Source Data
School of Transportation, Jilin University（吉林大学交通运输学院）；Department of Computer Science, City University of Hong Kong (Dongguan)（香港城市大学（东莞）计算机科学系）；Institute for Transport Planning and Systems, ETH Zurich（苏黎世联邦理工学院交通规划与系统研究所）；Institute of Intelligent Transportation Systems, College of Civil Engineering and Architecture, Zhejiang University（浙江大学智能交通系统研究所）；ZJU-UIUC Institute, Zhejiang University（浙江大学-UIUC研究院）
A Critical Synthesis of Uncertainty Quantification and Foundation Models in Monocular Depth Estimation
Institute of Photogrammetry and Remote Sensing (IPF), Karlsruhe Institute of Technology (KIT)（摄影测量与遥感研究所（卡尔斯鲁厄理工学院））；The Ohio State University（俄亥俄州立大学）
Imputation-free transformer learning enables robust Alzheimer's disease prediction and calibrated uncertainty quantification across heterogeneous clinical cohorts
Platform of Bioinformatics（生物信息平台）；Lausanne University Hospital（洛桑大学医院）；Faculty of Biology and Medicine（生物学与医学学院）；University of Lausanne（洛桑大学）；Leenaards Memory Centre（Leenaards记忆中心）；Department of Clinical Neurosciences（临床神经科学系）
Towards Trustworthy Vital Sign Forecasting: Leveraging Uncertainty for Prediction Intervals
College of Computing（计算学院）；Data Science Nanyang Technological University（南洋理工大学数据科学系）；Nanyang Technological University（南洋理工大学）；A*STAR Centre for Frontier AI Research（前沿人工智能研究中心）；School of Computing（计算学院）；University of Utah（犹他大学）；Swee Hock School of Public Health（公共卫生学院）；National University of Singapore（新加坡国立大学）；Lee Kong Chian School of Medicine（李科钦医学院）
Queen Elizabeth Hospital（伊丽莎白女王医院）；The Chinese University of Hong Kong（香港中文大学）
Brain midline shift (MLS) is one of the most critical factors to be considered for clinical diagnosis and treatment decision-making for intracranial hemorrhage. Existing computational methods on MLS quantification not only require intensive labeling in millimeter-level measurement but also suffer from poor performance due to their dependence on specific landmarks or simplified anatomical assumptions. In this paper, we propose a novel semi-supervised framework to accurately measure the scale of MLS from head CT scans. We formulate the MLS measurement task as a deformation estimation problem and…
Trustworthy deployment of ML models requires a proper measure of uncertainty, especially in safety-critical applications. We focus on uncertainty quantification (UQ) for classification problems via two avenues -- prediction sets using conformal prediction and calibration of probabilistic predictors by post-hoc binning -- since these possess distribution-free guarantees for i.i.d. data. Two common ways of generalizing beyond the i.i.d. setting include handling covariate and label shift. Within the context of distribution-free UQ, the former has already received attention, but not the latter. It…
A review of uncertainty quantification in medical image analysis: probabilistic and non-probabilistic methods
Saw Swee Hock School of Public Health, National University of Singapore（新加坡国立大学苏瑞福公共卫生学院）；University of Rouen Normandy（鲁昂诺曼底大学）；Institute of Data Science, National University of Singapore（新加坡国立大学数据科学研究所）
Accurate and trustworthy epidemic forecasting is an important problem that has impact on public health planning and disease mitigation. Most existing epidemic forecasting models disregard uncertainty quantification, resulting in mis-calibrated predictions. Recent works in deep neural models for uncertainty-aware time-series forecasting also have several limitations; e.g. it is difficult to specify meaningful priors in Bayesian NNs, while methods like deep ensembling are computationally expensive in practice. In this paper, we fill this important gap. We model the forecasting task as a probabil…
A biosignal is a signal that can be continuously measured from human bodies, such as respiratory sounds, heart activity (ECG), brain waves (EEG), etc, based on which, machine learning models have been developed with very promising performance for automatic disease detection and health status monitoring. However, dataset shift, i.e., data distribution of inference varies from the distribution of the training, is not uncommon for real biosignal-based applications. To improve the robustness, probabilistic models with uncertainty quantification are adapted to capture how reliable a prediction is. …
Evidential time-to-event prediction with calibrated uncertainty quantification
Saw Swee Hock School of Public Health, National University of Singapore（新加坡国立大学苏瑞福公共卫生学院）；Institute of Data Science, National University of Singapore（新加坡国立大学数据科学研究所）；Université de Technologie de Compiègne, CNRS, Heudiasyc（贡比涅技术大学、法国国家科研中心、Heudiasyc实验室）；Institut universitaire de France（法国大学研究院）
Ensuring trustworthiness in machine learning (ML) models is a multi-dimensional task. In addition to the traditional notion of predictive performance, other notions such as privacy, fairness, robustness to distribution shift, adversarial robustness, interpretability, explainability, and uncertainty quantification are important considerations to evaluate and improve (if deficient). However, these sub-disciplines or 'pillars' of trustworthiness have largely developed independently, which has limited us from understanding their interactions in real-world ML pipelines. In this paper, focusing spec…
OREHAS: A fully automated deep-learning pipeline for volumetric endolymphatic hydrops quantification in MRI
Bioengineering Department, Universidad Carlos III de Madrid（卡洛斯三世大学生物工程系）；Radiology Department, Clínica Universidad de Navarra（纳瓦拉大学医院放射科）；Otorhinolaryngology Department, Clínica Universidad de Navarra（纳瓦拉大学医院耳鼻喉科）
Medical College of Wisconsin（威斯康星医学院）；Marquette University（马凯特大学）
Quantitative susceptibility mapping (QSM) utilizes MRI signal phase to infer estimates of local tissue magnetism (magnetic susceptibility), which has been shown useful to provide novel image contrast and as biomarkers of abnormal tissue. QSM requires addressing a challenging post-processing problem: filtering of image phase estimates and inversion of the phase to susceptibility relationship. A wide variety of quantification errors, robustness limitations, and artifacts plague QSM algorithms. To overcome these limitations, a robust deep-learning-based single-step QSM reconstruction approach is …
