Weakly Supervised Anomaly Detection for Chest X-Ray Image
School of Artificial Intelligence, Beijing University of Posts and Telecommunications（北京邮电大学人工智能学院）；College of Information and Engineering, Capital Normal University（首都师范大学信息与工程学院）；Peking University Third Hospital（北京大学第三医院）
Haoqi Ni, Ximiao Zhang, Min Xu, Ning Lang, Xiuzhuang Zhou
Chest X-Ray (CXR) examination is a common method for assessing thoracic diseases in clinical applications. While recent advances in deep learning have enhanced the significance of visual analysis for CXR anomaly detection, current methods often miss key cues in anomaly images crucial for identifying disease regions, as they predominantly rely on unsupervised training with normal images. This letter focuses on a more practical setup in which few-shot anomaly images with only image-level labels are available during training. For this purpose, we propose WSCXR, a weakly supervised anomaly detection framework for CXR. WSCXR firstly constructs sets of normal and anomaly image features respectively. It then refines the anomaly image features by eliminating normal region features through anomaly feature mining, thus fully leveraging the scarce yet crucial features of diseased areas. Additionally, WSCXR employs a linear mixing strategy to augment the anomaly features, facilitating the training of anomaly detector with few-shot anomaly images. Experiments on two CXR datasets demonstrate the effectiveness of our approach.
Northwestern Polytechnical University（西北工业大学）；Research & Development Institute of Northwestern Polytechnical University in Shenzhen（西北工业大学深圳研究院）；The University of Adelaide（阿德莱德大学）；GreyBird Ventures, LLC（GreyBird Ventures有限责任公司）；JF Healthcare Inc.（JF Healthcare公司）
Cluster of viral pneumonia occurrences during a short period of time may be a harbinger of an outbreak or pandemic, like SARS, MERS, and recent COVID-19. Rapid and accurate detection of viral pneumonia using chest X-ray can be significantly useful in large-scale screening and epidemic prevention, particularly when other chest imaging modalities are less available. Viral pneumonia often have diverse causes and exhibit notably different visual appearances on X-ray images. The evolution of viruses and the emergence of novel mutated viruses further result in substantial dataset shift, which greatl…
The Hong Kong University of Science and Technology（香港科技大学）；Huazhong University of Science and Technology（华中科技大学）
Chest X-ray (CXR) is the most typical radiological exam for diagnosis of various diseases. Due to the expensive and time-consuming annotations, detecting anomalies in CXRs in an unsupervised fashion is very promising. However, almost all of the existing methods consider anomaly detection as a one-class classification (OCC) problem. They model the distribution of only known normal images during training and identify the samples not conforming to normal profile as anomalies in the testing phase. A large number of unlabeled images containing anomalies are thus ignored in the training phase, altho…
Toronto Rehabilitation Institute, University Health Network（多伦多康复研究所，大学健康网络）；University of Toronto（多伦多大学）；University of Manitoba（曼尼托巴大学）
The current COVID-19 pandemic is now getting contained, albeit at the cost of morethan2.3million human lives. A critical phase in any pandemic is the early detection of cases to develop preventive treatments and strategies. In the case of COVID-19,several studies have indicated that chest radiography images of the infected patients show characteristic abnormalities. However, at the onset of a given pandemic, such asCOVID-19, there may not be sufficient data for the affected cases to train models for their robust detection. Hence, supervised classification is ill-posed for this problem because …
AMAE: Adaptation of Pre-Trained Masked Autoencoder for Dual-Distribution Anomaly Detection in Chest X-Rays
École Polytechnique Fédérale de Lausanne (EPFL)（洛桑联邦理工学院（EPFL））；Lausanne University Hospital (CHUV)（洛桑大学医院（CHUV））；Inception Institute of AI (IIAI)（Inception人工智能研究院（IIAI））
HIDSS4Health - Helmholtz Information and Data Science School for Health（HIDSS4Health——亥姆霍兹健康信息与数据科学学院）；Karlsruhe Institute of Technology（卡尔斯鲁厄理工学院）；German Cancer Research Center Heidelberg（海德堡德国癌症研究中心）
The lack of fine-grained annotations hinders the deployment of automated diagnosis systems, which require human-interpretable justification for their decision process. In this paper, we address the problem of weakly supervised identification and localization of abnormalities in chest radiographs. To that end, we introduce a novel loss function for training convolutional neural networks increasing the \emph{localization confidence} and assisting the overall \emph{disease identification}. The loss leverages both image- and patch-level predictions to generate auxiliary supervision. Rather than fo…
Osaka University Graduate School of Medicine（大阪大学医学研究生院）
Large numbers of labeled medical images are essential for the accurate detection of anomalies, but manual annotation is labor-intensive and time-consuming. Self-supervised learning (SSL) is a training method to learn data-specific features without manual annotation. Several SSL-based models have been employed in medical image anomaly detection. These SSL methods effectively learn representations in several field-specific images, such as natural and industrial product images. However, owing to the requirement of medical expertise, typical SSL-based models are inefficient in medical image anomal…
The University of Texas at Arlington（德克萨斯大学阿灵顿分校）；Tencent AI Lab（腾讯人工智能实验室）
Chest X-rays is one of the most commonly available and affordable radiological examinations in clinical practice. While detecting thoracic diseases on chest X-rays is still a challenging task for machine intelligence, due to 1) the highly varied appearance of lesion areas on X-rays from patients of different thoracic disease and 2) the shortage of accurate pixel-level annotations by radiologists for model training. Existing machine learning methods are unable to deal with the challenge that thoracic diseases usually happen in localized disease-specific areas. In this article, we propose a weak…
EXACT: an explainable anomaly-aware vision foundation model for analysis of 3D chest CT
School of Biomedical Engineering, Tsinghua University（清华大学生物医学工程学院）；Department of Radiology, Mianyang Central Hospital（绵阳市中心医院放射科）；Center for Biomedical Imaging Research, Tsinghua University（清华大学生物医学成像研究中心）
Anomaly detection and localization in medical images is a challenging task, especially when the anomaly exhibits a change of existing structures, e.g., brain atrophy or changes in the pleural space due to pleural effusions. In this work, we present a weakly supervised and detail-preserving method that is able to detect structural changes of existing anatomical structures. In contrast to standard anomaly detection methods, our method extracts information about the disease characteristics from two groups: a group of patients affected by the same disease and a healthy control group. Together with…
Deep Convolutional Neural Networks have proven effective in solving the task of semantic segmentation. However, their efficiency heavily relies on the pixel-level annotations that are expensive to get and often require domain expertise, especially in medical imaging. Weakly supervised semantic segmentation helps to overcome these issues and also provides explainable deep learning models. In this paper, we propose a novel approach to the semantic segmentation of medical chest X-ray images with only image-level class labels as supervision. We improve the disease localization accuracy by combinin…
We consider the problem of abnormality localization for clinical applications. While deep learning has driven much recent progress in medical imaging, many clinical challenges are not fully addressed, limiting its broader usage. While recent methods report high diagnostic accuracies, physicians have concerns trusting these algorithm results for diagnostic decision-making purposes because of a general lack of algorithm decision reasoning and interpretability. One potential way to address this problem is to further train these models to localize abnormalities in addition to just classifying them…
Anatomy-Structured Hierarchical MIL for Weakly-Supervised Thoracic Disease Detection in Chest X-rays
Ewha Womans University（梨花女子大学）；Ewha Medical Artificial Intelligence Research Institute（梨花女子大学医学人工智能研究所）；REMEDI Inc.（瑞美迪公司）；Ewha Womans University Seoul Hospital（梨花女子大学首尔医院）
National Institutes of Health, Clinical Center（美国国立卫生研究院临床中心）；Ping An Technology Co., Ltd.（平安科技有限公司）
In this work, we exploit the task of joint classification and weakly supervised localization of thoracic diseases from chest radiographs, with only image-level disease labels coupled with disease severity-level (DSL) information of a subset. A convolutional neural network (CNN) based attention-guided curriculum learning (AGCL) framework is presented, which leverages the severity-level attributes mined from radiology reports. Images in order of difficulty (grouped by different severity-levels) are fed to CNN to boost the learning gradually. In addition, highly confident samples (measured by cla…
The University of Texas at Austin（德克萨斯大学奥斯汀分校）；Weill Cornell Medical College（威尔康奈尔医学院）
Before the recent success of deep learning methods for automated medical image analysis, practitioners used handcrafted radiomic features to quantitatively describe local patches of medical images. However, extracting discriminative radiomic features relies on accurate pathology localization, which is difficult to acquire in real-world settings. Despite advances in disease classification and localization from chest X-rays, many approaches fail to incorporate clinically-informed domain knowledge. For these reasons, we propose a Radiomics-Guided Transformer (RGT) that fuses \textit{global} image…
Given image labels as the only supervisory signal, we focus on harvesting, or mining, thoracic disease localizations from chest X-ray images. Harvesting such localizations from existing datasets allows for the creation of improved data sources for computer-aided diagnosis and retrospective analyses. We train a convolutional neural network (CNN) for image classification and propose an attention mining (AM) strategy to improve the model's sensitivity or saliency to disease patterns. The intuition of AM is that once the most salient disease area is blocked or hidden from the CNN model, it will pa…
Carnegie Mellon University（卡内基梅隆大学）；University of Pennsylvania（宾夕法尼亚大学）
We present a weakly supervised deep learning model for classifying thoracic diseases and identifying abnormalities in chest radiography. In this work, instead of learning from medical imaging data with region-level annotations, our model was merely trained on imaging data with image-level labels to classify diseases, and is able to identify abnormal image regions simultaneously. Our model consists of a customized pooling structure and an adaptive DenseNet front-end, which can effectively recognize possible disease features for classification and localization tasks. Our method has been validate…
Nanjing University（南京大学）；Drum Tower Hospital, Nanjing University（南京大学医学院附属鼓楼医院）；Chinese Academy of Medical Sciences and Peking Union Medical College（中国医学科学院北京协和医学院）
The chest X-Ray (CXR) is the one of the most common clinical exam used to diagnose thoracic diseases and abnormalities. The volume of CXR scans generated daily in hospitals is huge. Therefore, an automated diagnosis system able to save the effort of doctors is of great value. At present, the applications of artificial intelligence in CXR diagnosis usually use pattern recognition to classify the scans. However, such methods rely on labeled databases, which are costly and usually have large error rates. In this work, we built a database containing more than 12,000 CXR scans and radiological repo…
National Center for Biotechnology Information (NCBI)（美国国家生物技术信息中心）；National Library of Medicine (NLM)（美国国家医学图书馆）；National Institutes of Health (NIH)（美国国立卫生研究院）；Weill Cornell Medicine（威尔康奈尔医学院）；University of Texas at Arlington（德克萨斯大学阿灵顿分校）
The latest threat to global health is the COVID-19 outbreak. Although there exist large datasets of chest X-rays (CXR) and computed tomography (CT) scans, few COVID-19 image collections are currently available due to patient privacy. At the same time, there is a rapid growth of COVID-19-relevant articles in the biomedical literature. Here, we present COVID-19-CT-CXR, a public database of COVID-19 CXR and CT images, which are automatically extracted from COVID-19-relevant articles from the PubMed Central Open Access (PMC-OA) Subset. We extracted figures, associated captions, and relevant figure…
University of Pittsburgh（匹兹堡大学）；University of Pittsburgh Medical Center（匹兹堡大学医学中心）
Creating a large-scale dataset of abnormality annotation on medical images is a labor-intensive and costly task. Leveraging weak supervision from readily available data such as radiology reports can compensate lack of large-scale data for anomaly detection methods. However, most of the current methods only use image-level pathological observations, failing to utilize the relevant anatomy mentions in reports. Furthermore, Natural Language Processing (NLP)-mined weak labels are noisy due to label sparsity and linguistic ambiguity. We propose an Anatomy-Guided chest X-ray Network (AGXNet) to addr…
MILA（蒙特利尔学习算法研究所）；Université de Montréal（蒙特利尔大学）；Imagia Inc.（Imagia公司）；School of Computing Science, Simon Fraser University（西蒙弗雷泽大学计算科学学院）
The scarcity of richly annotated medical images is limiting supervised deep learning based solutions to medical image analysis tasks, such as localizing discriminatory radiomic disease signatures. Therefore, it is desirable to leverage unsupervised and weakly supervised models. Most recent weakly supervised localization methods apply attention maps or region proposals in a multiple instance learning formulation. While attention maps can be noisy, leading to erroneously highlighted regions, it is not simple to decide on an optimal window/bag size for multiple instance learning approaches. In th…
Stanford University（斯坦福大学）；University of California, Berkeley（加州大学伯克利分校）；IBM Almaden Research Center（IBM阿尔马登研究中心）
Chest X-rays have become the focus of vigorous deep learning research in recent years due to the availability of large labeled datasets. While classification of anomalous findings is now possible, ensuring that they are correctly localized still remains challenging, as this requires recognition of anomalies within anatomical regions. Existing deep learning networks for fine-grained anomaly classification learn location-specific findings using architectures where the location and spatial contiguity information is lost during the flattening step before classification. In this paper, we present a…
The increased availability and accuracy of eye-gaze tracking technology has sparked attention-related research in psychology, neuroscience, and, more recently, computer vision and artificial intelligence. The attention mechanism in artificial neural networks is known to improve learning tasks. However, no previous research has combined the network attention and human attention. This paper describes a gaze-guided class activation mapping (GG-CAM) method to directly regulate the formation of network attention based on expert radiologists' visual attention for the chest X-ray pathology classifica…
We propose a data collecting and annotation pipeline that extracts information from Vietnamese radiology reports to provide accurate labels for chest X-ray (CXR) images. This can benefit Vietnamese radiologists and clinicians by annotating data that closely match their endemic diagnosis categories which may vary from country to country. To assess the efficacy of the proposed labeling technique, we built a CXR dataset containing 9,752 studies and evaluated our pipeline using a subset of this dataset. With an F1-score of at least 0.9923, the evaluation demonstrates that our labeling tool perform…
Technion - Israel Institute of Technology（以色列理工学院）；Google Research（谷歌研究院）
Localization of an object within an image is a common task in medical imaging. Learning to localize or detect objects typically requires the collection of data which has been labelled with bounding boxes or similar annotations, which can be very time consuming and expensive. A technique which could perform such learning with much less annotation would, therefore, be quite valuable. We present such a technique for localization with limited annotation, in which the number of images with bounding boxes can be a small fraction of the total dataset (e.g. less than 1%); all other images only possess…
Christian Brothers University（基督教兄弟大学）；University of Dayton（代顿大学）
With a Coronavirus disease (COVID-19) case count exceeding 10 million worldwide, there is an increased need for a diagnostic capability. The main variables in increasing diagnostic capability are reduced cost, turnaround or diagnosis time, and upfront equipment cost and accessibility. Two candidates for machine learning COVID-19 diagnosis are Computed Tomography (CT) scans and plain chest X-rays. While CT scans score higher in sensitivity, they have a higher cost, maintenance requirement, and turnaround time as compared to plain chest X-rays. The use of portable chest X-radiograph (CXR) is rec…
University of Central Florida（中佛罗里达大学）；Mitsubishi Electric Research Laboratories（三菱电机研究实验室）；Siemens Corporate Technology（西门子企业技术）
Anomaly localization is an important problem in computer vision which involves localizing anomalous regions within images with applications in industrial inspection, surveillance, and medical imaging. This task is challenging due to the small sample size and pixel coverage of the anomaly in real-world scenarios. Most prior works need to use anomalous training images to compute a class-specific threshold to localize anomalies. Without the need of anomalous training images, we propose Convolutional Adversarial Variational autoencoder with Guided Attention (CAVGA), which localizes the anomaly wit…
Image representation is a fundamental task in computer vision. However, most of the existing approaches for image representation ignore the relations between images and consider each input image independently. Intuitively, relations between images can help to understand the images and maintain model consistency over related images, leading to better explainability. In this paper, we consider modeling the image-level relations to generate more informative image representations, and propose ImageGCN, an end-to-end graph convolutional network framework for inductive multi-relational image modelin…
Deep Omni-supervised Learning for Rib Fracture Detection from Chest Radiology Images
Imsight AI Research Lab（Imsight AI 研究实验室）；The Hong Kong University of Science and Technology（香港科技大学）；HKUST Shenzhen-Hong Kong Collaborative Innovation Research Institute（香港科技大学深圳-香港协同创新研究院）；The Chinese University of Hong Kong（香港中文大学）
Towards Universal Unsupervised Anomaly Detection in Medical Imaging
Technical University of Munich（慕尼黑工业大学）；Helmholtz AI and Helmholtz Center Munich（亥姆霍兹人工智能与亥姆霍兹慕尼黑中心）；Klinikum Rechts der Isar（伊萨尔河右岸医院）；Imperial College London（伦敦帝国理工学院）
Pre-trained models, e.g., from ImageNet, have proven to be effective in boosting the performance of many downstream applications. It is too demanding to acquire large-scale annotations to build such models for medical imaging. Meanwhile, there are numerous clinical data (in the form of images and text reports) stored in the hospital information systems. The paired image-text data from the same patient study could be utilized for the pre-training task in a weakly supervised manner. However, the integrity, accessibility, and amount of such raw data vary across different institutes, e.g., paired …
Philips Research（飞利浦研究院）；Skolkovo Institute of Science and Technology（斯科尔科沃科学技术研究院）；Goethe University Frankfurt（法兰克福歌德大学）
Anomaly detection is the problem of recognizing abnormal inputs based on the seen examples of normal data. Despite recent advances of deep learning in recognizing image anomalies, these methods still prove incapable of handling complex medical images, such as barely visible abnormalities in chest X-rays and metastases in lymph nodes. To address this problem, we introduce a new powerful method of image anomaly detection. It relies on the classical autoencoder approach with a re-designed training pipeline to handle high-resolution, complex images and a robust way of computing an image abnormalit…
Instance level detection of thoracic diseases or abnormalities are crucial for automatic diagnosis in chest X-ray images. Most existing works on chest X-rays focus on disease classification and weakly supervised localization. In order to push forward the research on disease classification and localization on chest X-rays. We provide a new benchmark called ChestX-Det10, including box-level annotations of 10 categories of disease/abnormality of $\sim$ 3,500 images. The annotations are located at https://github.com/Deepwise-AILab/ChestX-Det10-Dataset.
The University of Tokyo Hospital（东京大学医院）；The University of Tokyo（东京大学）；RIKEN（理化学研究所）
Oversight in medical images is a crucial problem, and timely reporting of medical images is desired. Therefore, an all-purpose anomaly detection method that can detect virtually all types of lesions/diseases in a given image is strongly desired. However, few commercially available and versatile anomaly detection methods for medical images have been provided so far. Recently, anomaly detection methods built upon deep learning methods have been rapidly growing in popularity, and these methods seem to provide reasonable solutions to the problem. However, the workload to label the images necessary…
The Hong Kong University of Science and Technology（香港科技大学）；Huazhong University of Science and Technology（华中科技大学）
Medical anomaly detection is a crucial yet challenging task aimed at recognizing abnormal images to assist in diagnosis. Due to the high-cost annotations of abnormal images, most methods utilize only known normal images during training and identify samples deviating from the normal profile as anomalies in the testing phase. Many readily available unlabeled images containing anomalies are thus ignored in the training phase, restricting the performance. To solve this problem, we introduce one-class semi-supervised learning (OC-SSL) to utilize known normal and unlabeled images for training, and p…
Australian Institute for Machine Learning, The University of Adelaide（阿德莱德大学澳大利亚机器学习研究所）；Institute for Systems and Robotics, Instituto Superior Tecnico（高等技术学院系统与机器人研究所）
The deployment of automated systems to diagnose diseases from medical images is challenged by the requirement to localise the diagnosed diseases to justify or explain the classification decision. This requirement is hard to fulfil because most of the training sets available to develop these systems only contain global annotations, making the localisation of diseases a weakly supervised approach. The main methods designed for weakly supervised disease classification and localisation rely on saliency or attention maps that are not specifically trained for localisation, or on region proposals tha…
In medical imaging, Class-Activation Map (CAM) serves as the main explainability tool by pointing to the region of interest. Since the localization accuracy from CAM is constrained by the resolution of the model's feature map, one may expect that segmentation models, which generally have large feature maps, would produce more accurate CAMs. However, we have found that this is not the case due to task mismatch. While segmentation models are developed for datasets with pixel-level annotation, only image-level annotation is available in most medical imaging datasets. Our experiments suggest that …
Subgroup performance analysis of adaptation strategies for chest X-ray foundation models
Imperial College London（帝国理工学院）；Royal Brompton Hospital（皇家布朗普顿医院）；Causality in Healthcare AI Hub（医疗保健AI因果关系中心）；National Heart & Lung Institute（国家心肺研究所）
Lung nodules are commonly missed in chest radiographs. We propose and evaluate P-AnoGAN, an unsupervised anomaly detection approach for lung nodules in radiographs. P-AnoGAN modifies the fast anomaly detection generative adversarial network (f-AnoGAN) by utilizing a progressive GAN and a convolutional encoder-decoder-encoder pipeline. Model training uses only unlabelled healthy lung patches extracted from the Indiana University Chest X-Ray Collection. External validation and testing are performed using healthy and unhealthy patches extracted from the ChestX-ray14 and Japanese Society for Radio…
From Classification to Localization and Clinical Validation: Large-Scale Development of a Deep Learning System for Thoracic Disease Detection on Chest Radiographs in Thailand
Perceptra Co., Ltd.（Perceptra有限公司）；Radiology Department, Faculty of Medicine Siriraj Hospital, Mahidol University（玛希隆大学诗里拉吉医院医学院放射科）；Institute of Field Robotics, King Mongkut’s University of Technology Thonburi（泰国国王科技大学吞武里分校野外机器人研究所）
cOOpD: Reformulating COPD classification on chest CT scans as anomaly detection using contrastive representations
German Cancer Research Center（德国癌症研究中心）；Heidelberg University（海德堡大学）；Translational Lung Research Center Heidelberg（海德堡转化肺研究中心）；German Center for Lung Research（德国肺研究中心）；Helmholtz Imaging（亥姆霍兹影像中心）
Samsung Medical Center（三星医疗中心）；Sungkyunkwan University School of Medicine（成均馆大学医学院）；Sungkyunkwan University（成均馆大学）
Computer-aided diagnosis for low-dose computed tomography (CT) based on deep learning has recently attracted attention as a first-line automatic testing tool because of its high accuracy and low radiation exposure. However, existing methods rely on supervised learning, imposing an additional burden to doctors for collecting disease data or annotating spatial labels for network training, consequently hindering their implementation. We propose a method based on a deep neural network for computer-aided diagnosis called virtual multi-view projection and reconstruction for unsupervised anomaly dete…
Despite the potential of weakly supervised learning to automatically annotate massive amounts of data, little is known about its limitations for use in computer-aided diagnosis (CAD). For CT specifically, interpreting the performance of CAD algorithms can be challenging given the large number of co-occurring diseases. This paper examines the effect of co-occurring diseases when training classification models by weakly supervised learning, specifically by comparing multi-label and multiple binary classifiers using the same training data. Our results demonstrated that the binary model outperform…
The University of Hong Kong（香港大学）；Xidian University（西安电子科技大学）；Sun Yat-Sen University（中山大学）；DeepWise AI Lab（深睿医疗人工智能实验室）
Identifying and locating diseases in chest X-rays are very challenging, due to the low visual contrast between normal and abnormal regions, and distortions caused by other overlapping tissues. An interesting phenomenon is that there exist many similar structures in the left and right parts of the chest, such as ribs, lung fields and bronchial tubes. This kind of similarities can be used to identify diseases in chest X-rays, according to the experience of broad-certificated radiologists. Aimed at improving the performance of existing detection methods, we propose a deep end-to-end module to exp…
In medical applications, weakly supervised anomaly detection methods are of great interest, as only image-level annotations are required for training. Current anomaly detection methods mainly rely on generative adversarial networks or autoencoder models. Those models are often complicated to train or have difficulties to preserve fine details in the image. We present a novel weakly supervised anomaly detection method based on denoising diffusion implicit models. We combine the deterministic iterative noising and denoising scheme with classifier guidance for image-to-image translation between d…
