Quantifying and Leveraging Predictive Uncertainty for Medical Image Assessment
Siemens Healthineers（西门子医疗）；University of Michigan（密歇根大学）；Massachusetts General Hospital（麻省总医院）；Harvard Medical School（哈佛医学院）
Florin C. Ghesu, Bogdan Georgescu, Awais Mansoor, Youngjin Yoo, Eli Gibson, R. S. Vishwanath, Abishek Balachandran, James M. Balter, Yue Cao, Ramandeep Singh, S… 展开作者
Florin C. Ghesu, Bogdan Georgescu, Awais Mansoor, Youngjin Yoo, Eli Gibson, R. S. Vishwanath, Abishek Balachandran, James M. Balter, Yue Cao, Ramandeep Singh, Subba R. Digumarthy, Mannudeep K. Kalra, Sasa Grbic, Dorin Comaniciu
The interpretation of medical images is a challenging task, often complicated by the presence of artifacts, occlusions, limited contrast and more. Most notable is the case of chest radiography, where there is a high inter-rater variability in the detection and classification of abnormalities. This is largely due to inconclusive evidence in the data or subjective definitions of disease appearance. An additional example is the classification of anatomical views based on 2D Ultrasound images. Often, the anatomical context captured in a frame is not sufficient to recognize the underlying anatomy. Current machine learning solutions for these problems are typically limited to providing probabilistic predictions, relying on the capacity of underlying models to adapt to limited information and the high degree of label noise. In practice, however, this leads to overconfident systems with poor generalization on unseen data. To account for this, we propose a system that learns not only the probabilistic estimate for classification, but also an explicit uncertainty measure which captures the confidence of the system in the predicted output. We argue that this approach is essential to account for the inherent ambiguity characteristic of medical images from different radiologic exams including computed radiography, ultrasonography and magnetic resonance imaging. In our experiments we demonstrate that sample rejection based on the predicted uncertainty can significantly improve the ROC-AUC for various tasks, e.g., by 8% to 0.91 with an expected rejection rate of under 25% for the classification of different abnormalities in chest radiographs. In addition, we show that using uncertainty-driven bootstrapping to filter the training data, one can achieve a significant increase in robustness and accuracy.
CONNECT-AI Research Center, Yonsei University College of Medicine（延世大学医学院CONNECT-AI研究中心）；Gordon Center for Medical Imaging, Massachusetts General Hospital（麻省总医院戈登医学影像中心）；Harvard Medical School（哈佛医学院）
Medical imaging, including MRI, CT, and Ultrasound, plays a vital role in clinical decisions. Accurate segmentation is essential to measure the structure of interest from the image. However, manual segmentation is highly operator-dependent, which leads to high inter and intra-variability of quantitative measurements. In this paper, we explore the feasibility that Bayesian predictive distribution parameterized by deep neural networks can capture the clinicians' inter-intra variability. By exploring and analyzing recently emerged approximate inference schemes, we evaluate whether approximate Bay…
Istanbul Technical University（伊斯坦布尔技术大学）；Mehmet Akif Ersoy Thoracic and Cardiovascular Surgery Training and Research Hospital（穆罕默德·阿基夫·埃尔索伊胸心血管外科培训研究医院）；Acibadem University（阿哲巴登大学）；King’s College London（伦敦国王学院）
To maintain a standard in a medical imaging study, images should have necessary image quality for potential diagnostic use. Although CNN-based approaches are used to assess the image quality, their performance can still be improved in terms of accuracy. In this work, we approach this problem by using Swin Transformer, which improves the poor-quality image classification performance that causes the degradation in medical image quality. We test our approach on Foreign Object Classification problem on Chest X-Rays (Object-CXR) and Left Ventricular Outflow Tract Classification problem on Cardiac M…
A study of why we need to reassess full reference image quality assessment with medical images
University of Cambridge（剑桥大学）；Medical University of Vienna（维也纳医科大学）；Emory University（埃默里大学）；Research Center for Clinical AI-Research in Omics and Medical Data Science (CAROM)（组学与医疗数据科学临床人工智能研究中心（CAROM））；Austrian Center for Medical Innovation and Technology（奥地利医疗创新与技术中心）；Harvard Medical School（哈佛医学院）；Brigham and Women’s Hospital（布莱根妇女医院）
California Institute of Technology（加州理工学院）；Stanford University（斯坦福大学）
Magnetic Resonance Imaging (MRI) suffers from several artifacts, the most common of which are motion artifacts. These artifacts often yield images that are of non-diagnostic quality. To detect such artifacts, images are prospectively evaluated by experts for their diagnostic quality, which necessitates patient-revisits and rescans whenever non-diagnostic quality scans are encountered. This motivates the need to develop an automated framework capable of accessing medical image quality and detecting diagnostic and non-diagnostic images. In this paper, we explore several convolutional neural netw…
Medical image quality assessment is an important aspect of image acquisition, as poor-quality images may lead to misdiagnosis. Manual labelling of image quality is a tedious task for population studies and can lead to misleading results. While much research has been done on automated analysis of image quality to address this issue, relatively little work has been done to explain the methodologies. In this work, we propose an explainable image quality assessment system and validate our idea on two different objectives which are foreign object detection on Chest X-Rays (Object-CXR) and Left Vent…
Deep neural networks are increasingly being used for the analysis of medical images. However, most works neglect the uncertainty in the model's prediction. We propose an uncertainty-aware deep kernel learning model which permits the estimation of the uncertainty in the prediction by a pipeline of a Convolutional Neural Network and a sparse Gaussian Process. Furthermore, we adapt different pre-training methods to investigate their impacts on the proposed model. We apply our approach to Bone Age Prediction and Lesion Localization. In most cases, the proposed model shows better performance compar…
SQA-SAM: Segmentation Quality Assessment for Medical Images Utilizing the Segment Anything Model
Nanjing University of Science and Technology（南京理工大学）；Digital Medical Research Center, School of Basic Medical Sciences, Fudan University（复旦大学基础医学院数字医学研究中心）；The Chinese University of Hong Kong（香港中文大学）；University of Notre Dame（圣母大学）
University of British Columbia（不列颠哥伦比亚大学）；Princeton University（普林斯顿大学）
High-performance deep learning methods typically rely on large annotated training datasets, which are difficult to obtain in many clinical applications due to the high cost of medical image labeling. Existing data assessment methods commonly require knowing the labels in advance, which are not feasible to achieve our goal of 'knowing which data to label.' To this end, we formulate and propose a novel and efficient data assessment strategy, EXponentiAl Marginal sINgular valuE (EXAMINE) score, to rank the quality of unlabeled medical image data based on their useful latent representations extrac…
Royal College of Art（皇家艺术学院）；University of Warwick（华威大学）
Computer-assisted multimodal training is an effective way of learning complex motor skills in various applications. In particular disciplines (eg. healthcare) incompetency in performing dexterous hands-on examinations (clinical palpation) may result in misdiagnosis of symptoms, serious injuries or even death. Furthermore, a high quality clinical examination can help to exclude significant pathology, and reduce time and cost of diagnosis by eliminating the need for unnecessary medical imaging. Medical palpation is used regularly as an effective preliminary diagnosis method all around the world …
Siemens Healthineers（西门子医疗）；Massachusetts General Hospital（麻省总医院）；Harvard Medical School（哈佛医学院）
The interpretation of chest radiographs is an essential task for the detection of thoracic diseases and abnormalities. However, it is a challenging problem with high inter-rater variability and inherent ambiguity due to inconclusive evidence in the data, limited data quality or subjective definitions of disease appearance. Current deep learning solutions for chest radiograph abnormality classification are typically limited to providing probabilistic predictions, relying on the capacity of learning models to adapt to the high degree of label noise and become robust to the enumerated causal fact…
A study on the adequacy of common IQA measures for medical images
University of Cambridge（剑桥大学）；Medical University of Vienna（维也纳医科大学）；Cambridge University Hospitals（剑桥大学医院）；Cancer Research UK Cambridge Institute（英国癌症研究剑桥研究院）
UbiQVision: Quantifying Uncertainty in XAI for Image Recognition
Center for Artificial Intelligence in Public Health Research (ZKI-PH), Robert Koch Institute（公共健康人工智能研究中心（ZKI-PH），罗伯特·科赫研究所）；Department of Mathematics and Computer Science, Freie Universität Berlin（数学与计算机科学系，柏林自由大学）
A No-Reference Medical Image Quality Assessment Method Based on Automated Distortion Recognition Technology: Application to Preprocessing in MRI-guided Radiotherapy
Beijing University of Posts and Telecommunications（北京邮电大学）；National Cancer Center/National Clinical Research Center for Cancer/Cancer Hospital, Chinese Academy of Medical Sciences and Peking Union Medical College（中国医学科学院北京协和医学院国家癌症中心/国家癌症临床医学研究中心/肿瘤医院）
Recent works have shown that deep neural networks can achieve super-human performance in a wide range of image classification tasks in the medical imaging domain. However, these works have primarily focused on classification accuracy, ignoring the important role of uncertainty quantification. Empirically, neural networks are often miscalibrated and overconfident in their predictions. This miscalibration could be problematic in any automatic decision-making system, but we focus on the medical field in which neural network miscalibration has the potential to lead to significant treatment errors.…
Recent advances in reconstruction methods for inverse problems leverage powerful data-driven models, e.g., deep neural networks. These techniques have demonstrated state-of-the-art performances for several imaging tasks, but they often do not provide uncertainty on the obtained reconstruction. In this work, we develop a scalable, data-driven, knowledge-aided computational framework to quantify the model uncertainty via Bayesian neural networks. The approach builds on, and extends deep gradient descent, a recently developed greedy iterative training scheme, and recasts it within a probabilistic…
Vanderbilt University（范德堡大学）；Vanderbilt University Medical Center（范德堡大学医学中心）；U.S. Department of Veterans Affairs（美国退伍军人事务部）
Image Quality Assessment (IQA) is important for scientific inquiry, especially in medical imaging and machine learning. Potential data quality issues can be exacerbated when human-based workflows use limited views of the data that may obscure digital artifacts. In practice, multiple factors such as network issues, accelerated acquisitions, motion artifacts, and imaging protocol design can impede the interpretation of image collections. The medical image processing community has developed a wide variety of tools for the inspection and validation of imaging data. Yet, IQA of computed tomography …
Improving Colorectal Cancer Screening and Risk Assessment through Predictive Modeling on Medical Images and Records
Geisel School of Medicine at Dartmouth（达特茅斯盖泽尔医学院）；Dartmouth-Hitchcock Medical Center（达特茅斯-希区柯克医学中心）；New Hampshire Colonoscopy Registry（新罕布什尔结肠镜检查登记处）；White River Junction VA Medical Center（怀特河交界处退伍军人事务医疗中心）；Dartmouth College（达特茅斯学院）
University of Guelph（圭尔夫大学）；Vector Institute（向量研究所）；Canadian Institute for Advanced Research（加拿大高等研究院）
The use of deep learning for medical imaging has seen tremendous growth in the research community. One reason for the slow uptake of these systems in the clinical setting is that they are complex, opaque and tend to fail silently. Outside of the medical imaging domain, the machine learning community has recently proposed several techniques for quantifying model uncertainty (i.e.~a model knowing when it has failed). This is important in practical settings, as we can refer such cases to manual inspection or correction by humans. In this paper, we aim to bring these recent results on estimating u…
Information Entropy-Based Framework for Quantifying Tortuosity in Meibomian Gland Uneven Atrophy
College of Mathematical Medicine, Zhejiang Normal University（数学医学学院，浙江师范大学）；National Clinical Research Center for Ocular Diseases, Eye Hospital, Wenzhou Medical University（眼病临床研究中心，温州医科大学眼医院）
Beyond Pixels: Medical Image Quality Assessment with Implicit Neural Representations
Computer Engineering Department, Istanbul Technical University（伊斯坦布尔技术大学计算机工程系）；Department of Applied Mathematics, Technical Medical Centre, University of Twente（代尔夫特理工大学应用数学系）
Computer Vision Laboratory, ETH Zürich（苏黎世联邦理工学院计算机视觉实验室）
Quantifying segmentation uncertainty has become an important issue in medical image analysis due to the inherent ambiguity of anatomical structures and its pathologies. Recently, neural network-based uncertainty quantification methods have been successfully applied to various problems. One of the main limitations of the existing techniques is the high memory requirement during training; which limits their application to processing smaller field-of-views (FOVs) and/or using shallower architectures. In this paper, we investigate the effect of using reversible blocks for building memory-efficient…
Quality assessment is a key element for the evaluation of hardware and software involved in image and video acquisition, processing, and visualization. In the medical field, user-based quality assessment is still considered more reliable than objective methods, which allow the implementation of automated and more efficient solutions. Regardless of increasing research in this topic in the last decade, defining quality standards for medical content remains a non-trivial task, as the focus should be on the diagnostic value assessed from expert viewers rather than the perceived quality from naïve …
Southern University of Science and Technology（南方科技大学）；The University of Hong Kong（香港大学）；The University of British Columbia（不列颠哥伦比亚大学）；Jiaxing Research Institute, Southern University of Science and Technology（南方科技大学嘉兴研究院）
Medical image quality assessment (MIQA) is a vital prerequisite in various medical image analysis applications. Most existing MIQA algorithms are fully supervised that request a large amount of annotated data. However, annotating medical images is time-consuming and labor-intensive. In this paper, we propose an unsupervised anomaly-aware framework with test-time clustering for optical coherence tomography angiography (OCTA) image quality assessment in a setting wherein only a set of high-quality samples are accessible in the training phase. Specifically, a feature-embedding-based low-quality r…
Université d'Orléans（奥尔良大学）；Institut National Des Télécommunications et TIC（国家电信与信息通信技术学院）
Predicting the quality of multimedia content is often needed in different fields. In some applications, quality metrics are crucial with a high impact, and can affect decision making such as diagnosis from medical multimedia. In this paper, we focus on such applications by proposing an efficient and shallow model for predicting the quality of medical images without reference from a small amount of annotated data. Our model is based on convolution self-attention that aims to model complex representation from relevant local characteristics of images, which itself slide over the image to interpol…
Boston Children’s Hospital（波士顿儿童医院）；Harvard Medical School（哈佛医学院）
Transfer learning is widely used for training machine learning models. Here, we study the role of transfer learning for training fully convolutional networks (FCNs) for medical image segmentation. Our experiments show that although transfer learning reduces the training time on the target task, the improvement in segmentation accuracy is highly task/data-dependent. Larger improvements in accuracy are observed when the segmentation task is more challenging and the target training data is smaller. We observe that convolutional filters of an FCN change little during training for medical image seg…
RIDGE: Reproducibility, Integrity, Dependability, Generalizability, and Efficiency Assessment of Medical Image Segmentation Models
University of Calgary（卡尔加里大学）；McGill University（麦吉尔大学）；University of Florida（佛罗里达大学）；New York University Langone Health（纽约大学朗格尼医学中心）；Mayo Clinic（梅奥诊所）；Carle College of Medicine University of Illinois Urbana-Champaign（伊利诺伊大学厄巴纳-香槟分校卡尔医学院）；Stanford School of Medicine（斯坦福医学院）；Mashhad University of Medical Sciences（马什哈德医科大学）；Children’s Hospital of Philadelphia（费城儿童医院）；Microsoft Research（微软研究院）；Diagnósticos da América S.A（美洲诊断公司）；DasaInova（达萨诺瓦）
University of Illinois at Urbana-Champaign（伊利诺伊大学厄巴纳-香槟分校）；Center for Devices and Radiological Health, Food and Drug Administration（美国食品药品监督管理局设备与放射健康中心）
In recent years, generative adversarial networks (GANs) have gained tremendous popularity for potential applications in medical imaging, such as medical image synthesis, restoration, reconstruction, translation, as well as objective image quality assessment. Despite the impressive progress in generating high-resolution, perceptually realistic images, it is not clear if modern GANs reliably learn the statistics that are meaningful to a downstream medical imaging application. In this work, the ability of a state-of-the-art GAN to learn the statistics of canonical stochastic image models (SIMs) t…
UQMIA: An Open, Hands-On Tutorial on Uncertainty Quantification in Medical Imaging Analysis with Large Language Model-Based Assessment of Educational Content
Mashhad University of Medical Sciences（马什哈德医科大学）；Northside Hospital（北院医院）；Mayo Clinic（梅奥诊所）；University of Pennsylvania（宾夕法尼亚大学）
AI WALKUP: A Computer-Vision Approach to Quantifying MDS-UPDRS in Parkinson's Disease
Key Lab of Image Processing and Intelligent Control, Ministry of Education（教育部图像处理与智能控制重点实验室）；School of Artificial Intelligence and Automation（人工智能与自动化学院）；Huazhong University of Science and Technology（华中科技大学）
The process of quantifying image quality consists of engineering the quality features and pooling these features to obtain a value or a map. There has been a significant research interest in designing the quality features but pooling is usually overlooked compared to feature design. In this work, we compare the state of the art quality and content-based spatial pooling strategies and show that although features are the key in any image quality assessment, pooling also matters. We also propose a quality-based spatial pooling strategy that is based on linearly weighted percentile pooling (WPP). …
