Calibrating Ensembles for Scalable Uncertainty Quantification in Deep Learning-based Medical Segmentation
University of Cambridge（剑桥大学）；Cancer Research UK Cambridge Centre（英国癌症研究中心剑桥中心）；Cancer Research UK Cambridge Institute（英国癌症研究中心剑桥研究所）；Danube Private University（多瑙河私立大学）；KTH Royal Institute of Technology（KTH皇家理工学院）
Thomas Buddenkotte, Lorena Escudero Sanchez, Mireia Crispin-Ortuzar, Ramona Woitek, Cathal McCague, James D. Brenton, Ozan Öktem, Evis Sala, Leonardo Rundo
Uncertainty quantification in automated image analysis is highly desired in many applications. Typically, machine learning models in classification or segmentation are only developed to provide binary answers; however, quantifying the uncertainty of the models can play a critical role for example in active learning or machine human interaction. Uncertainty quantification is especially difficult when using deep learning-based models, which are the state-of-the-art in many imaging applications. The current uncertainty quantification approaches do not scale well in high-dimensional real-world problems. Scalable solutions often rely on classical techniques, such as dropout, during inference or training ensembles of identical models with different random seeds to obtain a posterior distribution. In this paper, we show that these approaches fail to approximate the classification probability. On the contrary, we propose a scalable and intuitive framework to calibrate ensembles of deep learning models to produce uncertainty quantification measurements that approximate the classification probability. On unseen test data, we demonstrate improved calibration, sensitivity (in two out of three cases) and precision when being compared with the standard approaches. We further motivate the usage of our method in active learning, creating pseudo-labels to learn from unlabeled images and human-machine collaboration.
Univ. Grenoble Alpes（格勒诺布尔阿尔卑斯大学）；Inserm（法国国家健康与医学研究院）；Pixyl Research and Development Laboratory（Pixyl研发实验室）；Inria（法国国家信息与自动化研究所）；CNRS（法国国家科学研究中心）；Grenoble INP（格勒诺布尔理工学院）
The full acceptance of Deep Learning (DL) models in the clinical field is rather low with respect to the quantity of high-performing solutions reported in the literature. Particularly, end users are reluctant to rely on the rough predictions of DL models. Uncertainty quantification methods have been proposed in the literature as a potential response to reduce the rough decision provided by the DL black box and thus increase the interpretability and the acceptability of the result by the final user. In this review, we propose an overview of the existing methods to quantify uncertainty associate…
Manipal Institute of Technology（马尼帕尔理工学院）；University of Groningen（格罗宁根大学）
Trusting the predictions of deep learning models in safety critical settings such as the medical domain is still not a viable option. Distentangled uncertainty quantification in the field of medical imaging has received little attention. In this paper, we study disentangled uncertainties in image to image translation tasks in the medical domain. We compare multiple uncertainty quantification methods, namely Ensembles, Flipout, Dropout, and DropConnect, while using CycleGAN to convert T1-weighted brain MRI scans to T2-weighted brain MRI scans. We further evaluate uncertainty behavior in the pre…
Imperial College Business School（帝国理工学院商学院）；Department of Statistics and Operations Research（统计与运筹学系）；Imperial College London（伦敦帝国理工学院）；UNC-Chapel Hill（北卡罗来纳大学教堂山分校）
Quantification of Uncertainty with Adversarial Models in Medical Image Segmentation
Institute of Artificial Intelligence, Center for Medical Data Science, Medical University of Vienna, Austria（人工智能研究所、医学数据科学中心、维也纳医学大学，奥地利）；Comprehensive Center for AI in Medicine, Medical University of Vienna, Austria（医学人工智能综合中心、维也纳医学大学，奥地利）；ELLIS Unit Linz, LIT AI Lab and Institute for Machine Learning, Johannes Kepler University Linz, Austria（林茨ELLIS单位、LIT人工智能实验室和机器学习研究所、林茨约瑟夫·冯·克拉夫特大学，奥地利）；Institute for Machine Learning, Johannes Kepler University Linz, Austria（机器学习研究所、林茨约瑟夫·冯·克拉夫特大学，奥地利）；Clinical Research Center for Medical AI, Johannes Kepler University Linz, Austria（医学人工智能临床研究中心、林茨约瑟夫·冯·克拉夫特大学，奥地利）
The regulatory approval and broad clinical deployment of medical AI have been hampered by the perception that deep learning models fail in unpredictable and possibly catastrophic ways. A lack of statistically rigorous uncertainty quantification is a significant factor undermining trust in AI results. Recent developments in distribution-free uncertainty quantification present practical solutions for these issues by providing reliability guarantees for black-box models on arbitrary data distributions as formally valid finite-sample prediction intervals. Our work applies these new uncertainty qua…
National Library of Medicine, National Institutes of Health（美国国立卫生研究院国家医学图书馆）；Stanford University（斯坦福大学）
In medical image classification tasks, it is common to find that the number of normal samples far exceeds the number of abnormal samples. In such class-imbalanced situations, reliable training of deep neural networks continues to be a major challenge. Under these circumstances, the predicted class probabilities may be biased toward the majority class. Calibration has been suggested to alleviate some of these effects. However, there is insufficient analysis explaining when and whether calibrating a model would be beneficial in improving performance. In this study, we perform a systematic analys…
Bringing deep neural networks (DNNs) into safety critical applications such as automated driving, medical imaging and finance, requires a thorough treatment of the model's uncertainties. Training deep neural networks is already resource demanding and so is also their uncertainty quantification. In this overview article, we survey methods that we developed to teach DNNs to be uncertain when they encounter new object classes. Additionally, we present training methods to learn from only a few labels with help of uncertainty quantification. Note that this is typically paid with a massive overhead …
Recent works have shown that deep neural networks can achieve super-human performance in a wide range of image classification tasks in the medical imaging domain. However, these works have primarily focused on classification accuracy, ignoring the important role of uncertainty quantification. Empirically, neural networks are often miscalibrated and overconfident in their predictions. This miscalibration could be problematic in any automatic decision-making system, but we focus on the medical field in which neural network miscalibration has the potential to lead to significant treatment errors.…
Emergence of artificial intelligence techniques in biomedical applications urges the researchers to pay more attention on the uncertainty quantification (UQ) in machine-assisted medical decision making. For classification tasks, prior studies on UQ are difficult to compare with each other, due to the lack of a unified quantitative evaluation metric. Considering that well-performing UQ models ought to know when the classification models act incorrectly, we design a new evaluation metric, area under Confidence-Classification Characteristic curves (AUCCC), to quantitatively evaluate the performan…
Deakin University（迪肯大学）；Shenzhen University（深圳大学）；Dibrugarh University（迪布鲁格尔大学）；Swinburne University of Technology（斯威本科技大学）；University of Oulu（奥卢大学）；Google Research（谷歌研究院）；University of Waterloo（滑铁卢大学）；Institute of Information Engineering, Chinese Academy of Sciences（中国科学院信息工程研究所）
Uncertainty quantification (UQ) plays a pivotal role in reduction of uncertainties during both optimization and decision making processes. It can be applied to solve a variety of real-world applications in science and engineering. Bayesian approximation and ensemble learning techniques are two most widely-used UQ methods in the literature. In this regard, researchers have proposed different UQ methods and examined their performance in a variety of applications such as computer vision (e.g., self-driving cars and object detection), image processing (e.g., image restoration), medical image analy…
Turner Institute for Brain and Mental Health, Monash University（莫纳什大学特纳大脑与心理健康研究所）；Wellcome Centre for Human Neuroimaging, UCL（伦敦大学学院惠康人类神经成像中心）；Information Technology University (ITU) Punjab（旁遮普信息技术大学）
In recent years, deep learning (DL) techniques have provided state-of-the-art performance on different medical imaging tasks. However, the availability of good quality annotated medical data is very challenging due to involved time constraints and the availability of expert annotators, e.g., radiologists. In addition, DL is data-hungry and their training requires extensive computational resources. Another problem with DL is their black-box nature and lack of transparency on its inner working which inhibits causal understanding and reasoning. In this paper, we jointly address these challenges b…
Deep Learning methods are known to suffer from calibration issues: they typically produce over-confident estimates. These problems are exacerbated in the low data regime. Although the calibration of probabilistic models is well studied, calibrating extremely over-parametrized models in the low-data regime presents unique challenges. We show that deep-ensembles do not necessarily lead to improved calibration properties. In fact, we show that standard ensembling methods, when used in conjunction with modern techniques such as mixup regularization, can lead to less calibrated models. This text ex…
Institute for Intelligent Systems Research and Innovation (IISRI), Deakin University（迪肯大学智能系统研究与创新研究所（IISRI））；Omid hospital, Iran University of Medical Sciences（伊朗医科大学奥米德医院）；National University of Singapore（新加坡国立大学）
Deep neural networks (DNNs) have been widely applied for detecting COVID-19 in medical images. Existing studies mainly apply transfer learning and other data representation strategies to generate accurate point estimates. The generalization power of these networks is always questionable due to being developed using small datasets and failing to report their predictive confidence. Quantifying uncertainties associated with DNN predictions is a prerequisite for their trusted deployment in medical settings. Here we apply and evaluate three uncertainty quantification techniques for COVID-19 detecti…
University of Amsterdam（阿姆斯特丹大学）；Radboud University Medical Center（拉德堡德大学医学中心）
The accurate estimation of predictive uncertainty carries importance in medical scenarios such as lung node segmentation. Unfortunately, most existing works on predictive uncertainty do not return calibrated uncertainty estimates, which could be used in practice. In this work we exploit multi-grader annotation variability as a source of 'groundtruth' aleatoric uncertainty, which can be treated as a target in a supervised learning problem. We combine this groundtruth uncertainty with a Probabilistic U-Net and test on the LIDC-IDRI lung nodule CT dataset and MICCAI2012 prostate MRI dataset. We f…
Detection of Out-of-Distribution (OOD) samples in real time is a crucial safety check for deployment of machine learning models in the medical field. Despite a growing number of uncertainty quantification techniques, there is a lack of evaluation guidelines on how to select OOD detection methods in practice. This gap impedes implementation of OOD detection methods for real-world applications. Here, we propose a series of practical considerations and tests to choose the best OOD detector for a specific medical dataset. These guidelines are illustrated on a real-life use case of Electronic Healt…
ETH Zurich（苏黎世联邦理工学院）；Aimbrain ltd.（Aimbrain有限公司）；HeartFlow Inc.（HeartFlow公司）；King’s College London（伦敦国王学院）；Microsoft（微软）
Surface reconstruction is a vital tool in a wide range of areas of medical image analysis and clinical research. Despite the fact that many methods have proposed solutions to the reconstruction problem, most, due to their deterministic nature, do not directly address the issue of quantifying uncertainty associated with their predictions. We remedy this by proposing a novel probabilistic deep learning approach capable of simultaneous surface reconstruction and associated uncertainty prediction. The method incorporates prior shape information in the form of a principal component analysis (PCA) m…
Causal Attribution of Model Performance Gaps in Medical Imaging Under Distribution Shifts
CIBM Center for Biomedical Imaging（生物医学成像中心）；Department of Radiology, Lausanne University Hospital and University of Lausanne（拉沃大学医院放射科）；MedGIFT, Institute of Informatics, School of Management, HES–SO Valais–Wallis University of Applied Sciences and Arts Western Switzerland（MedGIFT，信息学院，管理学院，瓦莱-瓦莱大学应用科学和艺术西瑞士）
Word-Sequence Entropy: Towards Uncertainty Estimation in Free-Form Medical Question Answering Applications and Beyond
University of Electronic Science and Technology of China（电子科技大学）；Drexel University（德雷塞尔大学）；University of Pennsylvania（宾夕法尼亚大学）；Yale University（耶鲁大学）；Massachusetts Institute of Technology（麻省理工学院）；Illinois Institute of Technology（伊利诺伊理工学院）
Eigenvalue Calibration for Semantic Embeddings of Large Language Models
KU Leuven（鲁汶大学）；German Cancer Research Center (DKFZ)（德国癌症研究中心）；German Cancer Consortium (DKTK)（德国癌症联盟）；Goethe University Frankfurt（法兰克福歌德大学）；PSL Research University / Inria（巴黎文理研究大学/法国国家信息与自动化研究所）；Frankfurt Cancer Institute（法兰克福癌症研究所）
Department of Computer Science and Engineering, University of California San Diego, La Jolla, California, USA（加州大学圣地亚哥分校计算机科学与工程系，拉贾尔，加利福尼亚州，美国）；Halıcıoğlu Data Science Institute, University of California San Diego, La Jolla, California, USA（加州大学圣地亚哥分校Halıcıoğlu数据科学研究所，拉贾尔，加利福尼亚州，美国）；Department of Statistics, Stanford University, Stanford, California, USA（斯坦福大学统计学系，斯坦福，加利福尼亚州，美国）
CONNECT-AI Research Center, Yonsei University College of Medicine（延世大学医学院CONNECT-AI研究中心）；Gordon Center for Medical Imaging, Massachusetts General Hospital（麻省总医院戈登医学影像中心）；Harvard Medical School（哈佛医学院）
Medical imaging, including MRI, CT, and Ultrasound, plays a vital role in clinical decisions. Accurate segmentation is essential to measure the structure of interest from the image. However, manual segmentation is highly operator-dependent, which leads to high inter and intra-variability of quantitative measurements. In this paper, we explore the feasibility that Bayesian predictive distribution parameterized by deep neural networks can capture the clinicians' inter-intra variability. By exploring and analyzing recently emerged approximate inference schemes, we evaluate whether approximate Bay…
Centre for Intelligent Machines, McGill University（麦吉尔大学智能机器中心）
Although deep learning (DL) models have shown great success in many medical image analysis tasks, deployment of the resulting models into real clinical contexts requires: (1) that they exhibit robustness and fairness across different sub-populations, and (2) that the confidence in DL model predictions be accurately expressed in the form of uncertainties. Unfortunately, recent studies have indeed shown significant biases in DL models across demographic subgroups (e.g., race, sex, age) in the context of medical image analysis, indicating a lack of fairness in the models. Although several methods…
University of Florida（佛罗里达大学）；Tufts University（塔夫茨大学）
Deep neural networks (DNNs) have achieved tremendous success in computer vision, natural language processing, and scientific and engineering domains. However, DNNs can make unexpected, incorrect, yet overconfident predictions, leading to serious consequences in high-stakes applications such as autonomous driving, medical diagnosis, and disaster response. Uncertainty quantification (UQ) estimates the confidence of DNN predictions in addition to their accuracy. In recent years, many UQ methods have been developed for DNNs. It is valuable to systematically categorize these methods and compare the…
University of Massachusetts, Amherst（马萨诸塞大学阿默斯特分校）
We present an information-based uncertainty quantification method for general Markov Random Fields. Markov Random Fields (MRF) are structured, probabilistic graphical models over undirected graphs, and provide a fundamental unifying modeling tool for statistical mechanics, probabilistic machine learning, and artificial intelligence. Typically MRFs are complex and high-dimensional with nodes and edges (connections) built in a modular fashion from simpler, low-dimensional probabilistic models and their local connections; in turn, this modularity allows to incorporate available data to MRFs and e…
Black-box machine learning models are now routinely used in high-risk settings, like medical diagnostics, which demand uncertainty quantification to avoid consequential model failures. Conformal prediction is a user-friendly paradigm for creating statistically rigorous uncertainty sets/intervals for the predictions of such models. Critically, the sets are valid in a distribution-free sense: they possess explicit, non-asymptotic guarantees even without distributional assumptions or model assumptions. One can use conformal prediction with any pre-trained model, such as a neural network, to produ…
Autonomous Uncertainty Quantification for Computational Point-of-care Sensors
Electrical & Computer Engineering Department（电气与计算机工程系）；Bioengineering Department（生物工程系）；California NanoSystems Institute (CNSI)（加州纳米系统研究所）；Department of Surgery（外科医学系）；University of California, Los Angeles（加州大学洛杉矶分校）
Quantifying uncertainty in medical image segmentation applications is essential, as it is often connected to vital decision-making. Compelling attempts have been made in quantifying the uncertainty in image segmentation architectures, e.g. to learn a density segmentation model conditioned on the input image. Typical work in this field restricts these learnt densities to be strictly Gaussian. In this paper, we propose to use a more flexible approach by introducing Normalizing Flows (NFs), which enables the learnt densities to be more complex and facilitate more accurate modeling for uncertainty…
This work evaluates deep learning-based myocardial infarction (MI) quantification using Segment cardiovascular magnetic resonance (CMR) software. Segment CMR software incorporates the expectation-maximization, weighted intensity, a priori information (EWA) algorithm used to generate the infarct scar volume, infarct scar percentage, and microvascular obstruction percentage. Here, Segment CMR software segmentation algorithm is updated with semantic segmentation with U-net to achieve and evaluate fully automated or deep learning-based MI quantification. The direct observation of graphs and the nu…
Lawrence Livermore National Laboratory（劳伦斯利弗莫尔国家实验室）；IBM Research AI（IBM研究AI部门）；Arizona State University（亚利桑那州立大学）
The wide-spread adoption of representation learning technologies in clinical decision making strongly emphasizes the need for characterizing model reliability and enabling rigorous introspection of model behavior. While the former need is often addressed by incorporating uncertainty quantification strategies, the latter challenge is addressed using a broad class of interpretability techniques. In this paper, we argue that these two objectives are not necessarily disparate and propose to utilize prediction calibration to meet both objectives. More specifically, our approach is comprised of a ca…
Medical image segmentation is inherently an ambiguous task due to factors such as partial volumes and variations in anatomical definitions. While in most cases the segmentation uncertainty is around the border of structures of interest, there can also be considerable inter-rater differences. The class of conditional variational autoencoders (cVAE) offers a principled approach to inferring distributions over plausible segmentations that are conditioned on input images. Segmentation uncertainty estimated from samples of such distributions can be more informative than using pixel level probabilit…
We introduce a framework for calibrating machine learning models so that their predictions satisfy explicit, finite-sample statistical guarantees. Our calibration algorithms work with any underlying model and (unknown) data-generating distribution and do not require model refitting. The framework addresses, among other examples, false discovery rate control in multi-label classification, intersection-over-union control in instance segmentation, and the simultaneous control of the type-1 error of outlier detection and confidence set coverage in classification or regression. Our main insight is …
Beyond scalar losses: calibrating segmentation models via gradient vector field surgery
School of Computation, Information and Technology, TUM（慕尼黑工业大学计算、信息与技术学院）；Munich Center for Machine Learning（慕尼黑机器学习中心）；Department of Radiology, Weill Cornell Medicine（威尔康乃尔医学院放射科）；School of Medicine and Health, TUM University Hospital（慕尼黑工业大学医院医学与健康学院）；Cornell Tech（康奈尔科技学院）；Department of Computing, Imperial College London（伦敦帝国理工学院计算系）
