Improving Pneumonia Localization via Cross-Attention on Medical Images and Reports
School of Computing, University of Utah（犹他大学计算机学院）；NVIDIA Corporation（英伟达公司）；Department of Radiology and Imaging Sciences, National Institutes of Health Clinical Center（美国国立卫生研究院临床中心放射科与影像科学部）；National Institutes of Health Clinical Center（美国国立卫生研究院临床中心）；National Institutes of Health（美国国立卫生研究院）
Riddhish Bhalodia, Ali Hatamizadeh, Leo Tam, Ziyue Xu, Xiaosong Wang, Evrim Turkbey, Daguang Xu
Localization and characterization of diseases like pneumonia are primary steps in a clinical pipeline, facilitating detailed clinical diagnosis and subsequent treatment planning. Additionally, such location annotated datasets can provide a pathway for deep learning models to be used for downstream tasks. However, acquiring quality annotations is expensive on human resources and usually requires domain expertise. On the other hand, medical reports contain a plethora of information both about pneumonia characteristics and its location. In this paper, we propose a novel weakly-supervised attention-driven deep learning model that leverages encoded information in medical reports during training to facilitate better localization. Our model also performs classification of attributes that are associated to pneumonia and extracted from medical reports for supervision. Both the classification and localization are trained in conjunction and once trained, the model can be utilized for both the localization and characterization of pneumonia using only the input image. In this paper, we explore and analyze the model using chest X-ray datasets and demonstrate qualitatively and quantitatively that the introduction of textual information improves pneumonia localization. We showcase quantitative results on two datasets, MIMIC-CXR and Chest X-ray-8, and we also showcase severity characterization on the COVID-19 dataset.
Cockrell School of Engineering, The University of Texas at Austin（德克萨斯大学奥斯汀分校科克雷尔工程学院）；School of Information, The University of Texas at Austin（德克萨斯大学奥斯汀分校信息学院）；Weill Cornell Medicine（威尔康奈尔医学院）
Chest X-ray becomes one of the most common medical diagnoses due to its noninvasiveness. The number of chest X-ray images has skyrocketed, but reading chest X-rays still have been manually performed by radiologists, which creates huge burnouts and delays. Traditionally, radiomics, as a subfield of radiology that can extract a large number of quantitative features from medical images, demonstrates its potential to facilitate medical imaging diagnosis before the deep learning era. With the rise of deep learning, the explainability of deep neural networks on chest X-ray diagnosis remains opaque. …
Around 450 million people are affected by pneumonia every year which results in 2.5 million deaths. Covid-19 has also affected 181 million people which has lead to 3.92 million casualties. The chances of death in both of these diseases can be significantly reduced if they are diagnosed early. However, the current methods of diagnosing pneumonia (complaints + chest X-ray) and covid-19 (RT-PCR) require the presence of expert radiologists and time, respectively. With the help of Deep Learning models, pneumonia and covid-19 can be detected instantly from Chest X-rays or CT scans. This way, the pro…
The reliability of machine learning models can be compromised when trained on low quality data. Many large-scale medical imaging datasets contain low quality labels extracted from sources such as medical reports. Moreover, images within a dataset may have heterogeneous quality due to artifacts and biases arising from equipment or measurement errors. Therefore, algorithms that can automatically identify low quality data are highly desired. In this study, we used data Shapley, a data valuation metric, to quantify the value of training data to the performance of a pneumonia detection algorithm in…
Deep Learning-Based Pneumonia Detection from Chest X-ray Images: A CNN Approach with Performance Analysis and Clinical Implications
Amity Institute of Bio Technology, Amity University Kolkata（阿米蒂大学加尔各答分校阿米蒂生物技术学院）；Dept of Computer Science, Heritage Institute of Technology, Kolkata（加尔各答遗产技术学院计算机科学系）；School of Engineering and Technology, Amity University Kolkata（阿米蒂大学加尔各答分校工程与技术学院）
The University of Tokyo（东京大学）；The University of Tokyo Hospital（东京大学医院）
A computer-aided detection (CAD) system based on machine learning is expected to assist radiologists in making a diagnosis. It is desirable to build CAD systems for the various types of diseases accumulating daily in a hospital. An obstacle in developing a CAD system for a disease is that the number of medical images is typically too small to improve the performance of the machine learning model. In this paper, we aim to explore ways to address this problem through a sim2real transfer approach in medical image fields. To build a platform to evaluate the performance of sim2real transfer methods…
Mohammed VI Polytechnic University（穆罕默德六世理工大学）；Moulay Ismail University（穆莱·伊斯梅尔大学）；University Mohammed V in Rabat（拉巴特穆罕默德五世大学）；ENSIAS（国立统计与应用经济学院）
Recently, researchers, specialists, and companies around the world are rolling out deep learning and image processing-based systems that can fastly process hundreds of X-Ray and computed tomography (CT) images to accelerate the diagnosis of pneumonia such as SARS, COVID-19, and aid in its containment. Medical images analysis is one of the most promising research areas, it provides facilities for diagnosis and making decisions of a number of diseases such as MERS, COVID-19. In this paper, we present a comparison of recent Deep Convolutional Neural Network (DCNN) architectures for automatic bina…
In this work, we describe our approach to pneumonia classification and localization in chest radiographs. This method uses only \emph{open-source} deep learning object detection and is based on CoupleNet, a fully convolutional network which incorporates global and local features for object detection. Our approach achieves robustness through critical modifications of the training process and a novel ensembling algorithm which merges bounding boxes from several models. We tested our detection algorithm tested on a dataset of 3000 chest radiographs as part of the 2018 RSNA Pneumonia Challenge; ou…
The world is still overwhelmed by the spread of the COVID-19 virus. With over 250 Million infected cases as of November 2021 and affecting 219 countries and territories, the world remains in the pandemic period. Detecting COVID-19 using the deep learning method on CT scan images can play a vital role in assisting medical professionals and decision authorities in controlling the spread of the disease and providing essential support for patients. The convolution neural network is widely used in the field of large-scale image recognition. The current method of RT-PCR to diagnose COVID-19 is time-…
Advancing Diagnostic Precision: Leveraging Machine Learning Techniques for Accurate Detection of Covid-19, Pneumonia, and Tuberculosis in Chest X-Ray Images
Jadavpur University（贾达普大学）；Heritage Institute of Technology（赫里蒂奇理工学院）
Pneumonia is one of the major reasons for child mortality especially in income-deprived regions of the world. Although it can be detected and treated with very less sophisticated instruments and medication, Pneumonia detection still remains a major concern in developing countries. Computer-aided based diagnosis (CAD) systems can be used in such countries due to their lower operating costs than professional medical experts. In this paper, we propose a CAD system for Pneumonia detection from Chest X-rays, using the concepts of deep learning and a meta-heuristic algorithm. We first extract deep f…
With the advancement in AI, deep learning techniques are widely used to design robust classification models in several areas such as medical diagnosis tasks in which it achieves good performance. In this paper, we have proposed the CNN model (Convolutional Neural Network) for the classification of Chest X-ray images for Radiological Society of North America Pneumonia (RSNA) datasets. The study also tries to achieve the same RSNA benchmark results using the limited computational resources by trying out various approaches to the methodologies that have been implemented in recent years. The propo…
Translating Inference-Time Control to Radiology Vision-Language Models: Activation Steering for Pneumonia Classification on Chest X-rays
Universidade Federal de São Paulo (UNIFESP)（圣保罗联邦大学）；Hospital Israelita Albert Einstein（以色列阿尔伯特·爱因斯坦医院）；Stanford University School of Medicine（斯坦福大学医学院）；DASA；University of Texas Southwestern Medical Center (UTSW)（德克萨斯大学西南医学中心）；Eden
Sharif University of Technology（谢里夫理工大学）；Tehran University of Medical Sciences（德黑兰医科大学）；Iran University of Medical Sciences（伊朗医科大学）
COVID-19 is a virus with high transmission rate that demands rapid identification of the infected patients to reduce the spread of the disease. The current gold-standard test, Reverse-Transcription Polymerase Chain Reaction (RT-PCR), has a high rate of false negatives. Diagnosing from CT-scan images as a more accurate alternative has the challenge of distinguishing COVID-19 from other pneumonia diseases. Artificial intelligence can help radiologists and physicians to accelerate the process of diagnosis, increase its accuracy, and measure the severity of the disease. We designed a new interpret…
Bangladesh University of Engineering and Technology（孟加拉国工程技术大学）；University of Cincinnati College of Medicine（辛辛那提大学医学院）；University of Cincinnati（辛辛那提大学）
The outbreak of novel coronavirus disease (COVID- 19) has claimed millions of lives and has affected all aspects of human life. This paper focuses on the application of deep learning (DL) models to medical imaging and drug discovery for managing COVID-19 disease. In this article, we detail various medical imaging-based studies such as X-rays and computed tomography (CT) images along with DL methods for classifying COVID-19 affected versus pneumonia. The applications of DL techniques to medical images are further described in terms of image localization, segmentation, registration, and classifi…
National Heart and Lung Institute, Imperial College London（帝国理工学院国家心肺研究所）；Cardiovascular Research Centre, Royal Brompton Hospital（皇家布朗普顿医院心血管研究中心）
Medical image classification and segmentation based on deep learning (DL) are emergency research topics for diagnosing variant viruses of the current COVID-19 situation. In COVID-19 computed tomography (CT) images of the lungs, ground glass turbidity is the most common finding that requires specialist diagnosis. Based on this situation, some researchers propose the relevant DL models which can replace professional diagnostic specialists in clinics when lacking expertise. However, although DL methods have a stunning performance in medical image processing, the limited datasets can be a challeng…
The worldwide spread of pneumonia caused by a novel coronavirus poses an unprecedented challenge to the world's medical resources and prevention and control measures. Covid-19 attacks not only the lungs, making it difficult to breathe and life-threatening, but also the heart, kidneys, brain and other vital organs of the body, with possible sequela. At present, the detection of COVID-19 needs to be realized by the reverse transcription-polymerase Chain Reaction (RT-PCR). However, many countries are in the outbreak period of the epidemic, and the medical resources are very limited. They cannot p…
University of Patras（帕特雷大学）；Computer Technology Institute（计算机技术研究所）
In this study, a dataset of X-Ray images from patients with common pneumonia, Covid-19, and normal incidents was utilized for the automatic detection of the Coronavirus. The aim of the study is to evaluate the performance of state-of-the-art Convolutional Neural Network architectures proposed over recent years for medical image classification. Specifically, the procedure called transfer learning was adopted. With transfer learning, the detection of various abnormalities in small medical image datasets is an achievable target, often yielding remarkable results. The dataset utilized in this expe…
Does Bigger Mean Better? Comparitive Analysis of CNNs and Biomedical Vision Language Modles in Medical Diagnosis
University of Texas at Dallas（德克萨斯大学达拉斯分校）；Independent Researcher（独立研究者）；Duke University（杜克大学）；University of Michigan Ann Arbor（密歇根大学安娜堡分校）；Georgia Institute of Technology（佐治亚理工学院）；University of California, Irvine（加州大学 Irvine 分校）
In recent years, the prevalence of several pulmonary diseases, especially the coronavirus disease 2019 (COVID-19) pandemic, has attracted worldwide attention. These diseases can be effectively diagnosed and treated with the help of lung imaging. With the development of deep learning technology and the emergence of many public medical image datasets, the diagnosis of lung diseases via medical imaging has been further improved. This article reviews pulmonary CT and X-ray image detection and classification in the last decade. It also provides an overview of the detection of lung nodules, pneumoni…
Universidad de Granada（格拉纳达大学）；Universidad de Malaga（马拉加大学）
The ongoing crisis of the COVID-19 (Coronavirus disease 2019) pandemic has changed the world. According to the World Health Organization (WHO), 4 million people have died due to this disease, whereas there have been more than 180 million confirmed cases of COVID-19. The collapse of the health system in many countries has demonstrated the need of developing tools to automatize the diagnosis of the disease from medical imaging. Previous studies have used deep learning for this purpose. However, the performance of this alternative highly depends on the size of the dataset employed for training th…
University of Saskatchewan（萨斯喀彻温大学）；International Road Dynamics（国际道路动力学公司）；National Institute of Technology, Trichy（特里奇国家理工学院）
One of the primary clinical observations for screening the infectious by the novel coronavirus is capturing a chest x-ray image. In most of the patients, a chest x-ray contains abnormalities, such as consolidation, which are the results of COVID-19 viral pneumonia. In this study, research is conducted on efficiently detecting imaging features of this type of pneumonia using deep convolutional neural networks in a large dataset. It is demonstrated that simple models, alongside the majority of pretrained networks in the literature, focus on irrelevant features for decision-making. In this paper,…
Elsevier Science Publishers B.V.（爱思唯尔科学出版公司）；University of Winchester（温彻斯特大学）；Universidad de Granada（格拉纳达大学）；Universidad de Malaga（马拉加大学）；University of Leicester（莱斯特大学）
Respiratory diseases kill million of people each year. Diagnosis of these pathologies is a manual, time-consuming process that has inter and intra-observer variability, delaying diagnosis and treatment. The recent COVID-19 pandemic has demonstrated the need of developing systems to automatize the diagnosis of pneumonia, whilst Convolutional Neural Network (CNNs) have proved to be an excellent option for the automatic classification of medical images. However, given the need of providing a confidence classification in this context it is crucial to quantify the reliability of the model's predict…
University of Waterloo（滑铁卢大学）；Waterloo AI Institute（滑铁卢人工智能研究所）；Niagara Health（尼亚加拉医疗中心）；McMaster University（麦克马斯特大学）；Northern Ontario School of Medicine（北安大略医学院）；Thunder Bay Regional Health Sciences Centre（雷湾区域健康科学中心）
Medical image analysis continues to hold interesting challenges given the subtle characteristics of certain diseases and the significant overlap in appearance between diseases. In this work, we explore the concept of self-attention for tackling such subtleties in and between diseases. To this end, we introduce MEDUSA, a multi-scale encoder-decoder self-attention mechanism tailored for medical image analysis. While self-attention deep convolutional neural network architectures in existing literature center around the notion of multiple isolated lightweight attention mechanisms with limited indi…
University of Michigan（密歇根大学）；University of Michigan Medical School（密歇根大学医学院）
Objective: When patients develop acute respiratory failure, accurately identifying the underlying etiology is essential for determining the best treatment. However, differentiating between common medical diagnoses can be challenging in clinical practice. Machine learning models could improve medical diagnosis by aiding in the diagnostic evaluation of these patients. Materials and Methods: Machine learning models were trained to predict the common causes of acute respiratory failure (pneumonia, heart failure, and/or COPD). Models were trained using chest radiographs and clinical data from the e…
Technische Universität München（慕尼黑工业大学）；Klinikum rechts der Isar（伊萨尔河右岸医院）；TranslaTUM - Central Institute for Translational Cancer Research（慕尼黑转化癌症研究中心）；University of Zurich（苏黎世大学）；Helmholtz Zentrum München（慕尼黑亥姆霍兹中心）
Self-supervision has demonstrated to be an effective learning strategy when training target tasks on small annotated data-sets. While current research focuses on creating novel pretext tasks to learn meaningful and reusable representations for the target task, these efforts obtain marginal performance gains compared to fully-supervised learning. Meanwhile, little attention has been given to study the robustness of networks trained in a self-supervised manner. In this work, we demonstrate that networks trained via self-supervised learning have superior robustness and generalizability compared t…
Athinoula A. Martinos Center for Biomedical Imaging（阿西努拉·A·马蒂诺斯生物医学成像中心）；Massachusetts Institute of Technology（麻省理工学院）
While success of Deep Learning (DL) in automated diagnosis can be transformative to the medicinal practice especially for people with little or no access to doctors, its widespread acceptability is severely limited by inherent black-box decision making and unsafe failure modes. While saliency methods attempt to tackle this problem in non-medical contexts, their apriori explanations do not transfer well to medical usecases. With this study we validate a model design element agnostic to both architecture complexity and model task, and show how introducing this element gives an inherently self-ex…
Vanderbilt University（范德堡大学）；Vanderbilt University Medical Center（范德堡大学医学中心）
Atlas-based methods are the standard approaches for automatic targeting of the Anterior Nucleus of the Thalamus (ANT) for Deep Brain Stimulation (DBS), but these are known to lack robustness when anatomic differences between atlases and subjects are large. To improve the localization robustness, we propose a novel two-stage deep learning (DL) framework, where the first stage identifies and crops the thalamus regions from the whole brain MRI and the second stage performs per-voxel regression on the cropped volume to localize the targets at the finest resolution scale. To address the issue of da…
University of Waterloo（滑铁卢大学）；Vector Institute（向量研究所）
This paper introduces the `Projectron' as a new neural network architecture that uses Radon projections to both classify and represent medical images. The motivation is to build shallow networks which are more interpretable in the medical imaging domain. Radon transform is an established technique that can reconstruct images from parallel projections. The Projectron first applies global Radon transform to each image using equidistant angles and then feeds these transformations for encoding to a single layer of neurons followed by a layer of suitable kernels to facilitate a linear separation of…
Chest radiographs are used for the diagnosis of multiple critical illnesses (e.g., Pneumonia, heart failure, lung cancer), for this reason, systems for the automatic or semi-automatic analysis of these data are of particular interest. An efficient analysis of large amounts of chest radiographs can aid physicians and radiologists, ultimately allowing for better medical care of lung-, heart- and chest-related conditions. We propose a novel Discrete Wavelet Transform (DWT)-based method for the efficient identification and encoding of visual information that is typically lost in the down-sampling …
Deep learning technologies have already demonstrated a high potential to build diagnosis support systems from medical imaging data, such as Chest X-Ray images. However, the shortage of labeled data in the medical field represents one key obstacle to narrow down the performance gap with respect to applications in other image domains. In this work, we investigate the benefits of a curricular Self-Supervised Learning (SSL) pretraining scheme with respect to fully-supervised training regimes for pneumonia recognition on Chest X-Ray images of Covid-19 patients. We show that curricular SSL pretraini…
In 2019, the entire world is facing a situation of health emergency due to a newly emerged coronavirus (COVID-19). Almost 196 countries are affected by covid-19, while USA, Italy, China, Spain, Iran, and France have the maximum active cases of COVID-19. The issues, medical and healthcare departments are facing in delay of detecting the COVID-19. Several artificial intelligence based system are designed for the automatic detection of COVID-19 using chest x-rays. In this article we will discuss the different approaches used for the detection of COVID-19 and the challenges we are facing. It is ma…
