Self-supervised learning (SSL) approaches have recently shown substantial success in learning visual representations from unannotated images. Compared with photographic images, medical images acquired with the same imaging protocol exhibit high consistency in anatomy. To exploit this anatomical consistency, this paper introduces a novel SSL approach, called PEAC (patch embedding of anatomical consistency), for medical image analysis. Specifically, in this paper, we propose to learn global and local consistencies via stable grid-based matching, transfer pre-trained PEAC models to diverse downstream tasks, and extensively demonstrate that (1) PEAC achieves significantly better performance than the existing state-of-the-art fully/self-supervised methods, and (2) PEAC captures the anatomical structure consistency across views of the same patient and across patients of different genders, weights, and healthy statuses, which enhances the interpretability of our method for medical image analysis.
Pneumothorax is a relatively common disease, but in some cases, it may be difficult to find with chest radiography. In this paper, we propose a novel method of detecting pneumothorax in chest radiography. We propose an ensemble model of identical convolutional neural networks (CNN) with three different sizes of radiography images. Conventional methods may not properly characterize lost features while resizing large size images into 256 x 256 or 224 x 224 sizes. Our model is evaluated with ChestX-ray dataset which contains over 100,000 chest radiography images. As a result of the experiment, th…
Rutgers University（罗格斯大学）；Thomas Jefferson University（托马斯杰斐逊大学）
The purpose of this study is to develop an automated algorithm for thoracic vertebral segmentation on chest radiography using deep learning. 124 de-identified lateral chest radiographs on unique patients were obtained. Segmentations of visible vertebrae were manually performed by a medical student and verified by a board-certified radiologist. 74 images were used for training, 10 for validation, and 40 were held out for testing. A U-Net deep convolutional neural network was employed for segmentation, using the sum of dice coefficient and binary cross-entropy as the loss function. On the test s…
A Generative Foundation Model for Chest Radiography
Department of Radiation Oncology, Stanford University School of Medicine（斯坦福大学医学院放射肿瘤科）；Department of Computer Science, School of Computing and Data Science, The University of Hong Kong（香港大学计算机科学系）；Department of Population Medicine and Diagnostic Sciences, College of Veterinary Medicine, Cornell University（康奈尔大学兽医学院人口医学与诊断科学系）；Department of Radiology, The First Affiliated Hospital of Jinan University（暨南大学第一附属医院放射科）；Department of Radiology, Stanford University School of Medicine（斯坦福大学医学院放射科）；Department of Computer Science and Engineering, The Chinese University of Hong Kong（香港中文大学计算机科学与工程系）；Department of Biomedical Informatics, Harvard Medical School（哈佛医学院生物医学信息学系）
The University of Tokyo（东京大学）；NABLAS Inc.（NABLAS公司）
Chest radiography is a general method for diagnosing a patient's condition and identifying important information; therefore, radiography is used extensively in routine medical practice in various situations, such as emergency medical care and medical checkup. However, a high level of expertise is required to interpret chest radiographs. Thus, medical specialists spend considerable time in diagnosing such huge numbers of radiographs. In order to solve these problems, methods for generating findings have been proposed. However, the study of generating chest radiograph findings has primarily focu…
Chest radiography is the most common radiographic examination performed in daily clinical practice for the detection of various heart and lung abnormalities. The large amount of data to be read and reported, with more than 100 studies per day for a single radiologist, poses a challenge in consistently maintaining high interpretation accuracy. The introduction of large-scale public datasets has led to a series of novel systems for automated abnormality classification. However, the labels of these datasets were obtained using natural language processed medical reports, yielding a large degree of…
University of Waterloo（滑铁卢大学）；National Research Council Canada（加拿大国家研究委员会）
Building AI models with trustworthiness is important especially in regulated areas such as healthcare. In tackling COVID-19, previous work uses convolutional neural networks as the backbone architecture, which has shown to be prone to over-caution and overconfidence in making decisions, rendering them less trustworthy -- a crucial flaw in the context of medical imaging. In this study, we propose a feature learning approach using Vision Transformers, which use an attention-based mechanism, and examine the representation learning capability of Transformers as a new backbone architecture for medi…
Scaling Generative Foundation Models for Chest Radiography with Rectified Flow Transformers
Imperial College London（帝国理工学院）；Causality in Healthcare AI Hub（医疗AI因果关系中心）；University of Edinburgh（爱丁堡大学）；Cleveland Clinic London（克利夫兰诊所伦敦）；Department of Perioperative Medicine, CHU Clermont-Ferrand（克莱蒙费朗大学医院围手术期医学科）；Department of Medicine, Massachusetts General Hospital（麻省总医院医学部）；Broad Institute of MIT and Harvard（麻省理工学院与哈佛大学博德研究所）
Florida International University（佛罗里达国际大学）；Technical University of Munich（慕尼黑工业大学）；Stanford University（斯坦福大学）
Chest X-ray radiography is one of the earliest medical imaging technologies and remains one of the most widely-used for diagnosis, screening, and treatment follow up of diseases related to lungs and heart. The literature in this field of research reports many interesting studies dealing with the challenging tasks of bone suppression and organ segmentation but performed separately, limiting any learning that comes with the consolidation of parameters that could optimize both processes. This study, and for the first time, introduces a multitask deep learning model that generates simultaneously t…
XBench: A Comprehensive Benchmark for Visual-Language Explanations in Chest Radiography
ARTORG Center for Biomedical Engineering Research, University of Bern, Switzerland（ARTORG生物医学工程研究中心，伯尔尼大学，瑞士）；Shanghai Jiao Tong University, China（上海交通大学，中国）；Kaiko.AI, Switzerland（Kaiko.AI，瑞士）；Dept. of Radiation Oncology, Inselspital, Bern University Hospital（放射肿瘤科，因斯普尔茨医院，伯尔尼大学医院）
X2CT-CLIP: Enable Multi-Abnormality Detection in Computed Tomography from Chest Radiography via Tri-Modal Contrastive Learning
Peter Munk Cardiac Centre（彼得·芒克心脏中心）；University Health Network (UHN)（大学健康网络）；University of Toronto (U of T)（多伦多大学）；Ted Rogers Centre for Heart Research（泰德·罗杰斯心脏研究中心）；Toronto General Hospital Research Institute（多伦多综合医院研究所）；Vector Institute（矢量研究所）
Snap Inc.（Snap 公司）；Isfahan University of Medical Sciences（伊斯法罕医科大学）；The University of Iowa（爱荷华大学）；Isfahan University of Technology（伊斯法罕理工大学）
The COVID-19 pandemic is causing a major outbreak in more than 150 countries around the world, having a severe impact on the health and life of many people globally. One of the crucial step in fighting COVID-19 is the ability to detect the infected patients early enough, and put them under special care. Detecting this disease from radiography and radiology images is perhaps one of the fastest ways to diagnose the patients. Some of the early studies showed specific abnormalities in the chest radiograms of patients infected with COVID-19. Inspired by earlier works, we study the application of de…
Beyond the First Read: AI-Assisted Perceptual Error Detection in Chest Radiography Accounting for Interobserver Variability
High School Student（高中学生）；PhD Candidate（博士生）；Department of Electrical and Computer Engineering University of Houston（电气与计算机工程系，休斯顿大学）；Department of Computer Science Emory University（计算机科学系，埃默里大学）；Department of Thoracic Imaging, Division of Diagnostic Imaging The University of Texas MD Anderson Cancer Center, Houston, TX（胸腔成像部，诊断影像学分会，德克萨斯大学MD安德森癌症中心，休斯顿，TX）
National Institutes of Health (NIH) Clinical Center（美国国立卫生研究院临床中心）；Ping An Technology（平安科技）；Ping An Technology Co., Ltd.（平安科技有限公司）
Being one of the most common diagnostic imaging tests, chest radiography requires timely reporting of potential findings in the images. In this paper, we propose an end-to-end architecture for abnormal chest X-ray identification using generative adversarial one-class learning. Unlike previous approaches, our method takes only normal chest X-ray images as input. The architecture is composed of three deep neural networks, each of which learned by competing while collaborating among them to model the underlying content structure of the normal chest X-rays. Given a chest X-ray image in the testing…
Enhancing Representation in Radiography-Reports Foundation Model: A Granular Alignment Algorithm Using Masked Contrastive Learning
Paul C. Lauterbur Research Center for Biomedical Imaging（保罗·C·劳特伯生物医学成像研究中心）；Shenzhen Institute of Advanced Technology（深圳先进技术研究院）；Pengcheng Laboratory（鹏城实验室）；University of Chinese Academy of Sciences（中国科学院大学）；Harvard University（哈佛大学）；Shanghai AI Lab（上海人工智能实验室）
Computer-aided techniques may lead to more accurate and more acces-sible diagnosis of thorax diseases on chest radiography. Despite the success of deep learning-based solutions, this task remains a major challenge in smart healthcare, since it is intrinsically a weakly supervised learning problem. In this paper, we incorporate the attention mechanism into a deep convolutional neural network, and thus propose the ChestNet model to address effective diagnosis of thorax diseases on chest radiography. This model consists of two branches: a classification branch serves as a uniform feature extracti…
Seoul National University（首尔大学）；ASRI；INMC；ISRC；Institute of Engineering Research（工程研究所）
Automated diagnosis using deep neural networks in chest radiography can help radiologists detect life-threatening diseases. However, existing methods only provide predictions without accurate explanations, undermining the trustworthiness of the diagnostic methods. Here, we present XProtoNet, a globally and locally interpretable diagnosis framework for chest radiography. XProtoNet learns representative patterns of each disease from X-ray images, which are prototypes, and makes a diagnosis on a given X-ray image based on the patterns. It predicts the area where a sign of the disease is likely to…
Google Health（谷歌健康）；Advanced Clinical（先进临床）；Apollo Radiology International（阿波罗国际放射学）；Center of Infectious Disease Research in Zambia（赞比亚传染病研究中心）；Sibanye Stillwater（斯班耶-斯蒂尔沃特）；Clickmedix
Tuberculosis (TB) is a top-10 cause of death worldwide. Though the WHO recommends chest radiographs (CXRs) for TB screening, the limited availability of CXR interpretation is a barrier. We trained a deep learning system (DLS) to detect active pulmonary TB using CXRs from 9 countries across Africa, Asia, and Europe, and utilized large-scale CXR pretraining, attention pooling, and noisy student semi-supervised learning. Evaluation was on (1) a combined test set spanning China, India, US, and Zambia, and (2) an independent mining population in South Africa. Given WHO targets of 90% sensitivity an…
Carnegie Mellon University（卡内基梅隆大学）；University of Pennsylvania（宾夕法尼亚大学）
We present a weakly supervised deep learning model for classifying thoracic diseases and identifying abnormalities in chest radiography. In this work, instead of learning from medical imaging data with region-level annotations, our model was merely trained on imaging data with image-level labels to classify diseases, and is able to identify abnormal image regions simultaneously. Our model consists of a customized pooling structure and an adaptive DenseNet front-end, which can effectively recognize possible disease features for classification and localization tasks. Our method has been validate…
Technical University of Kosice（科希策技术大学）；Pavol Jozef Safarik University（帕沃尔·约瑟夫·沙法里克大学）；Louis Pasteur University Hospital（路易·巴斯德大学医院）
Chest radiography is a relatively cheap, widely available medical procedure that conveys key information for making diagnostic decisions. Chest X-rays are almost always used in the diagnosis of respiratory diseases such as pneumonia or the recent COVID-19. In this paper, we propose a self-supervised deep neural network that is pretrained on an unlabeled chest X-ray dataset. The learned representations are transferred to downstream task - the classification of respiratory diseases. The results obtained on four public datasets show that our approach yields competitive results without requiring l…
Epanechnikov nonparametric kernel density estimation based feature-learning in respiratory disease chest X-ray images
Department of Data Science, Data Science and AI Laboratory（数据科学系）；Catholic University of Argentina (UCA)（阿根廷天主教大学）；MACS School（MACS学院）；Heriot-Watt University（赫罗特-瓦特大学）
Less Could Be Better: Parameter-efficient Fine-tuning Advances Medical Vision Foundation Models
School of Informatics, Xiamen University（厦门大学信息学院）；Department of Biomedical Informatics, Harvard University（哈佛大学生物医学信息学系）；Department of Computer Science, The University of Hong Kong（香港大学计算机科学系）
本文实证评估 LoRA 对胸部 X 线摄影自监督基础模型的微调效果，证明其以不足 1% 参数在多数任务上超过全参数微调，并刷新数据高效学习基准。
National Research Council Canada（加拿大国家研究委员会）；University of Waterloo（滑铁卢大学）；Ford Motor Company（福特汽车公司）
Chest radiography is an effective screening tool for diagnosing pulmonary diseases. In computer-aided diagnosis, extracting the relevant region of interest, i.e., isolating the lung region of each radiography image, can be an essential step towards improved performance in diagnosing pulmonary disorders. Methods: In this work, we propose a deep learning approach to enhance abnormal chest x-ray (CXR) identification performance through segmentations. Our approach is designed in a cascaded manner and incorporates two modules: a deep neural network with criss-cross attention modules (XLSor) for loc…
One of the critical tools for early detection and subsequent evaluation of the incidence of lung diseases is chest radiography. This study presents a real-world implementation of a convolutional neural network (CNN) based Carebot Covid app to detect COVID-19 from chest X-ray (CXR) images. Our proposed model takes the form of a simple and intuitive application. Used CNN can be deployed as a STOW-RS prediction endpoint for direct implementation into DICOM viewers. The results of this study show that the deep learning model based on DenseNet and ResNet architecture can detect SARS-CoV-2 from CXR …
Long-Tailed Classification of Thorax Diseases on Chest X-Ray: A New Benchmark Study
The University of Texas at Austin（德克萨斯大学奥斯汀分校）；Texas A&M University（德克萨斯农工大学）；National Institutes of Health（美国国立卫生研究院）；Weill Cornell Medicine（威尔康奈尔医学院）
Anatomy-Structured Hierarchical MIL for Weakly-Supervised Thoracic Disease Detection in Chest X-rays
Ewha Womans University（梨花女子大学）；Ewha Medical Artificial Intelligence Research Institute（梨花女子大学医学人工智能研究所）；REMEDI Inc.（瑞美迪公司）；Ewha Womans University Seoul Hospital（梨花女子大学首尔医院）
CheXanatomy: Anatomy-Aware Vision-Language Modeling for Chest Radiographs
Stanford Center for Artificial Intelligence in Medicine and Imaging, Stanford University（斯坦福大学医学与影像人工智能中心）；Department of Radiology, Stanford University（斯坦福大学放射学系）
Prince Sultan University（苏丹王子大学）；University of Manouba（马努巴大学）
Deep learning (DL) is being increasingly utilized in healthcare-related fields due to its outstanding efficiency. However, we have to keep the individual health data used by DL models private and secure. Protecting data and preserving the privacy of individuals has become an increasingly prevalent issue. The gap between the DL and privacy communities must be bridged. In this paper, we propose privacy-preserving deep learning (PPDL)-based approach to secure the classification of Chest X-ray images. This study aims to use Chest X-ray images to their fullest potential without compromising the pri…
VRVis Zentrum für Virtual Reality und Visualisierung Forschungs-GmbH（VRVis虚拟现实与可视化研究中心有限公司）；Agfa NV（爱克发股份有限公司）；Antwerp University Hospital（安特卫普大学医院）；University of Antwerp（安特卫普大学）
To facilitate both the detection and the interpretation of findings in chest X-rays, comparison with a previous image of the same patient is very valuable to radiologists. Today, the most common approach for deep learning methods to automatically inspect chest X-rays disregards the patient history and classifies only single images as normal or abnormal. Nevertheless, several methods for assisting in the task of comparison through image registration have been proposed in the past. However, as we illustrate, they tend to miss specific types of pathological changes like cardiomegaly and effusion.…
Towards long-tailed, multi-label disease classification from chest X-ray: Overview of the CXR-LT challenge
The University of Texas at Austin（德克萨斯大学奥斯汀分校）；Weill Cornell Medicine（威尔康奈尔医学院）；Carnegie Mellon University（卡内基梅隆大学）；University of Science, VNU-HCM（越南胡志明市国家大学科学大学）；KT Research & Development Center, KT Corporation（KT公司研发中心）；Ajou University（亚洲大学）
National Cheng Kung University（国立成功大学）；National Cheng Kung University Hospital（国立成功大学附属医院）
The image quality of portable supine chest radiographs is inherently poor due to low contrast and high noise. The endotracheal intubation detection requires the locations of the endotracheal tube (ETT) tip and carina. The goal is to find the distance between the ETT tip and the carina in chest radiography. To overcome such a problem, we propose a feature extraction method with Mask R-CNN. The Mask R-CNN predicts a tube and a tracheal bifurcation in an image. Then, the feature extraction method is used to find the feature point of the ETT tip and that of the carina. Therefore, the ETT-carina di…
Mohamed bin Zayed University of Artificial Intelligence（穆罕默德·本·扎耶德人工智能大学）
Quick and accurate diagnosis is of paramount importance to mitigate the effects of COVID-19 infection, particularly for severe cases. Enormous effort has been put towards developing deep learning methods to classify and detect COVID-19 infections from chest radiography images. However, recently some questions have been raised surrounding the clinical viability and effectiveness of such methods. In this work, we investigate the impact of multi-task learning (classification and segmentation) on the ability of CNNs to differentiate between various appearances of COVID-19 infections in the lung. W…
Rochester Institute of Technology（罗切斯特理工学院）；IBM Almaden Research Center（IBM阿尔马登研究中心）
Chest radiography is the most common medical image examination for screening and diagnosis in hospitals. Automatic interpretation of chest X-rays at the level of an entry-level radiologist can greatly benefit work prioritization and assist in analyzing a larger population. Subsequently, several datasets and deep learning-based solutions have been proposed to identify diseases based on chest X-ray images. However, these methods are shown to be vulnerable to shift in the source of data: a deep learning model performing well when tested on the same dataset as training data, starts to perform poor…
Warsaw University of Technology（华沙理工大学）；Sano Centre for Computational Medicine（萨诺计算医学中心）；Jagiellonian University（雅盖隆大学）；Tooploox
A critical step in the fight against COVID-19, which continues to have a catastrophic impact on peoples lives, is the effective screening of patients presented in the clinics with severe COVID-19 symptoms. Chest radiography is one of the promising screening approaches. Many studies reported detecting COVID-19 in chest X-rays accurately using deep learning. A serious limitation of many published approaches is insufficient attention paid to explaining decisions made by deep learning models. Using explainable artificial intelligence methods, we demonstrate that model decisions may rely on confoun…
Exploring the Capabilities of Large Language Model Encoders for Image-Text Retrieval in Chest X-rays
Interdisciplinary Program in Bioengineering, Seoul National University Graduate School（生物工程跨学科项目，首尔国立大学研究生院）；Integrated Major in Innovative Medical Science, Seoul National University Graduate School（创新医学科学整合专业，首尔国立大学研究生院）；Department of Radiology, The First Affiliated Hospital, Zhejiang University School of Medicine（浙江大学医学院第一附属医院放射科）；Seoul National University College of Medicine（首尔国立大学医学院）；Department of Radiology, Seoul National University College of Medicine, Seoul National University Hospital（首尔国立大学医学院放射科，首尔国立大学医院）；Institute of Medical and Biological Engineering, Seoul National University Medical Research Center（医学与生物工程研究所，首尔国立大学医学研究所以及）；Institute of Radiation Medicine, Seoul National University Medical Research Center（放射医学研究所，首尔国立大学医学研究所以及）
Siemens Healthineers（西门子医疗）；University of Michigan（密歇根大学）
Building accurate and robust artificial intelligence systems for medical image assessment requires not only the research and design of advanced deep learning models but also the creation of large and curated sets of annotated training examples. Constructing such datasets, however, is often very costly -- due to the complex nature of annotation tasks and the high level of expertise required for the interpretation of medical images (e.g., expert radiologists). To counter this limitation, we propose a method for self-supervised learning of rich image features based on contrastive learning and onl…
UT Austin（得克萨斯大学奥斯汀分校）；MIS, Germany（德国曼海姆应用科学大学）；Weill Cornell Medicine（威尔康奈尔医学院）
Computer-aided diagnosis plays a salient role in more accessible and accurate cardiopulmonary diseases classification and localization on chest radiography. Millions of people get affected and die due to these diseases without an accurate and timely diagnosis. Recently proposed contrastive learning heavily relies on data augmentation, especially positive data augmentation. However, generating clinically-accurate data augmentations for medical images is extremely difficult because the common data augmentation methods in computer vision, such as sharp, blur, and crop operations, can severely alt…
University Medical Center Hamburg-Eppendorf（汉堡-埃彭多夫大学医学中心）；Hamburg University of Technology（汉堡工业大学）；Forschungszentrum Medizintechnik Hamburg（汉堡医学技术研究中心）；Philips Research（飞利浦研究院）
Chest radiography is the most common clinical examination type. To improve the quality of patient care and to reduce workload, methods for automatic pathology classification have been developed. In this contribution we investigate the usefulness of two advanced image pre-processing techniques, initially developed for image reading by radiologists, for the performance of Deep Learning methods. First, we use bone suppression, an algorithm to artificially remove the rib cage. Secondly, we employ an automatic lung field detection to crop the image to the lung area. Furthermore, we consider the com…
Microsoft Research（微软研究院）；University Hospitals Birmingham（伯明翰大学医院）
Chest radiography has been a recommended procedure for patient triaging and resource management in intensive care units (ICUs) throughout the COVID-19 pandemic. The machine learning efforts to augment this workflow have been long challenged due to deficiencies in reporting, model evaluation, and failure mode analysis. To address some of those shortcomings, we model radiological features with a human-interpretable class hierarchy that aligns with the radiological decision process. Also, we propose the use of a data-driven error analysis methodology to uncover the blind spots of our model, provi…
AnyCXR: Human Anatomy Segmentation of Chest X-ray at Any Acquisition Position using Multi-stage Domain Randomized Synthetic Data with Imperfect Annotations and Conditional Joint Annotation Regularization Learning
Department of Radiology, Northwestern University, Chicago, IL, USA（西北大学放射科）；Data Science Institute, Vanderbilt University, Nashville, TN, USA（范德比尔特大学数据科学研究院）；Department of Orthopedics, The Second Hospital of Shanxi Medical University, Taiyuan, Shanxi, P.R. China（山西医科大学第二医院骨科）；Second Clinical Medical College, Shanxi Medical University, Taiyuan, Shanxi, P.R. China（山西医科大学第二临床医学院）；School of Computer Science, University of Sydney, Sydney, NSW, Australia（悉尼大学计算机科学学院）
Beyond Diagnosis: Evaluating Multimodal LLMs for Pathology Localization in Chest Radiographs
University of California（加州大学）；Dana-Farber Cancer Institute（达纳-法伯癌症研究所）；Massachusetts General Hospital（麻省总医院）；St. Jude Children’s Research Hospital（圣犹大儿童研究医院）；Brigham and Women’s Hospital & Harvard Medical School（布里法伦医院及哈佛医学院）
Dartmouth College（达特茅斯学院）；Dartmouth-Hitchcock Medical Center（达特茅斯-希区柯克医疗中心）
Background: Deep learning has great potential to assist with detecting and triaging critical findings such as pneumoperitoneum on medical images. To be clinically useful, the performance of this technology still needs to be validated for generalizability across different types of imaging systems. Materials and Methods: This retrospective study included 1,287 chest X-ray images of patients who underwent initial chest radiography at 13 different hospitals between 2011 and 2019. The chest X-ray images were labelled independently by four radiologist experts as positive or negative for pneumoperito…
