Automatic medical report generation from chest X-ray images is one possibility for assisting doctors to reduce their workload. However, the different patterns and data distribution of normal and abnormal cases can bias machine learning models. Previous attempts did not focus on isolating the generation of the abnormal and normal sentences in order to increase the variability of generated paragraphs. To address this, we propose to separate abnormal and normal sentence generation by using two different word LSTMs in a hierarchical LSTM model. We conduct an analysis on the distinctiveness of generated sentences compared to the BLEU score, which increases when less distinct reports are generated. We hope our findings will help to encourage the development of new metrics to better verify methods of automatic medical report generation.
The Chinese University of Hong Kong（香港中文大学）；The Hong Kong University of Science and Technology（香港科技大学）
Deep learning models were frequently reported to learn from shortcuts like dataset biases. As deep learning is playing an increasingly important role in the modern healthcare system, it is of great need to combat shortcut learning in medical data as well as develop unbiased and trustworthy models. In this paper, we study the problem of developing debiased chest X-ray diagnosis models from the biased training data without knowing exactly the bias labels. We start with the observations that the imbalance of bias distribution is one of the key reasons causing shortcut learning, and the dataset bi…
DUCX: Decomposing Unfairness in Tool-Using Chest X-ray Agents
Institute of Artificial Intelligence, Hefei Comprehensive National Science Center, Anhui, China（人工智能研究所，合肥国家科学中心，安徽，中国）；The University of British Columbia, Vancouver, BC V6Z 1Z4, Canada（不列颠哥伦比亚大学，温哥华，BC V6Z 1Z4，加拿大）；Vector Institute, Toronto, ON M5G 1M1, Canada（向量研究所，多伦多，ON M5G 1M1，加拿大）
Warsaw University of Technology（华沙理工大学）；Sano Centre for Computational Medicine（萨诺计算医学中心）；Jagiellonian University（雅盖隆大学）；Tooploox
A critical step in the fight against COVID-19, which continues to have a catastrophic impact on peoples lives, is the effective screening of patients presented in the clinics with severe COVID-19 symptoms. Chest radiography is one of the promising screening approaches. Many studies reported detecting COVID-19 in chest X-rays accurately using deep learning. A serious limitation of many published approaches is insufficient attention paid to explaining decisions made by deep learning models. Using explainable artificial intelligence methods, we demonstrate that model decisions may rely on confoun…
Addressing Asynchronicity in Clinical Multimodal Fusion via Individualized Chest X-ray Generation
School of Nursing, The Hong Kong Polytechnic University（香港理工大学护理学院）；School of Software Engineering, South China University of Technology（华南理工大学软件学院）；Department of Computer Science, Hong Kong Baptist University（香港浸会大学计算机科学系）
针对 EHR 与 CXR 采集不同步导致旧胸片影响临床预测的问题，提出 DDL-CXR，利用既往 CXR 和 EHR 时间序列条件化潜在扩散模型生成最新患者特异性表示，并在 MIMIC 上验证其性能优势。
Automatic extraction of medical conditions from free-text radiology reports is critical for supervising computer vision models to interpret medical images. In this work, we show that radiologists labeling reports significantly disagree with radiologists labeling corresponding chest X-ray images, which reduces the quality of report labels as proxies for image labels. We develop and evaluate methods to produce labels from radiology reports that have better agreement with radiologists labeling images. Our best performing method, called VisualCheXbert, uses a biomedically-pretrained BERT model to …
Deep neural networks for image-based screening and computer-aided diagnosis have achieved expert-level performance on various medical imaging modalities, including chest radiographs. Recently, several works have indicated that these state-of-the-art classifiers can be biased with respect to sensitive patient attributes, such as race or gender, leading to growing concerns about demographic disparities and discrimination resulting from algorithmic and model-based decision-making in healthcare. Fair machine learning has focused on mitigating such biases against disadvantaged or marginalised group…
We consider the problem of abnormality localization for clinical applications. While deep learning has driven much recent progress in medical imaging, many clinical challenges are not fully addressed, limiting its broader usage. While recent methods report high diagnostic accuracies, physicians have concerns trusting these algorithm results for diagnostic decision-making purposes because of a general lack of algorithm decision reasoning and interpretability. One potential way to address this problem is to further train these models to localize abnormalities in addition to just classifying them…
AlphaRAD: Grounded Zero-Shot Classification in Chest Radiology via $α$-Corrected Binary Cross Entropy and Factorized Latent Supervision
Peter Munk Cardiac Centre, University Health Network (UHN)（彼得·芒克心脏中心，大学健康网络（UHN））；Ted Rogers Centre for Heart Research（泰德·罗杰斯心脏研究中心）；University of Toronto (U of T)（多伦多大学）；Vector Institute（向量研究所）；Toronto General Hospital Research Institute, UHN（多伦多总医院研究所，大学健康网络（UHN））
Clinical Center（临床中心）；National Library of Medicine（美国国家医学图书馆）；National Institutes of Health（美国国立卫生研究院）
Despite the recent advances in automatically describing image contents, their applications have been mostly limited to image caption datasets containing natural images (e.g., Flickr 30k, MSCOCO). In this paper, we present a deep learning model to efficiently detect a disease from an image and annotate its contexts (e.g., location, severity and the affected organs). We employ a publicly available radiology dataset of chest x-rays and their reports, and use its image annotations to mine disease names to train convolutional neural networks (CNNs). In doing so, we adopt various regularization tech…
Long-Tailed Classification of Thorax Diseases on Chest X-Ray: A New Benchmark Study
The University of Texas at Austin（德克萨斯大学奥斯汀分校）；Texas A&M University（德克萨斯农工大学）；National Institutes of Health（美国国立卫生研究院）；Weill Cornell Medicine（威尔康奈尔医学院）
The chest X-ray plays a key role in screening and diagnosis of many lung diseases including the COVID-19. More recently, many works construct deep neural networks (DNNs) for chest X-ray images to realize automated and efficient diagnosis of lung diseases. However, bias field caused by the improper medical image acquisition process widely exists in the chest X-ray images while the robustness of DNNs to the bias field is rarely explored, which definitely poses a threat to the X-ray-based automated diagnosis system. In this paper, we study this problem based on the recent adversarial attack and p…
Anatomy-Structured Hierarchical MIL for Weakly-Supervised Thoracic Disease Detection in Chest X-rays
Ewha Womans University（梨花女子大学）；Ewha Medical Artificial Intelligence Research Institute（梨花女子大学医学人工智能研究所）；REMEDI Inc.（瑞美迪公司）；Ewha Womans University Seoul Hospital（梨花女子大学首尔医院）
There are multiple papers published about different AI models for the COVID-19 diagnosis with promising results. Unfortunately according to the reviews many of the papers do not reach the level of sophistication needed for a clinically usable model. In this paper I go through multiple review papers, guidelines, and other relevant material in order to generate more comprehensive requirements for the future papers proposing a AI based diagnosis of the COVID-19 from chest X-ray data (CXR). Main findings are that a clinically usable AI needs to have an extremely good documentation, comprehensive s…
RadJEPA: Radiology Encoder for Chest X-Rays via Joint Embedding Predictive Architecture
Department of Computer Science and Engineering, Indian Institute of Technology Bombay（印度理工学院孟买分校计算机科学与工程系）；Department of Biomedical Engineering, Johns Hopkins University（约翰霍普金斯大学生物医学工程系）；Koita Centre for Digital Health, Indian Institute of Technology Bombay（印度理工学院孟买分校Koita数字健康中心）
Massachusetts Institute of Technology（麻省理工学院）；University of Toronto（多伦多大学）；University of Tübingen（蒂宾根大学）；University of Adelaide（阿德莱德大学）；Stanford University（斯坦福大学）
Deep learning models have reached or surpassed human-level performance in the field of medical imaging, especially in disease diagnosis using chest x-rays. However, prior work has found that such classifiers can exhibit biases in the form of gaps in predictive performance across protected groups. In this paper, we question whether striving to achieve zero disparities in predictive performance (i.e. group fairness) is the appropriate fairness definition in the clinical setting, over minimax fairness, which focuses on maximizing the performance of the worst-case group. We benchmark the performan…
A Generative Foundation Model for Chest Radiography
Department of Radiation Oncology, Stanford University School of Medicine（斯坦福大学医学院放射肿瘤科）；Department of Computer Science, School of Computing and Data Science, The University of Hong Kong（香港大学计算机科学系）；Department of Population Medicine and Diagnostic Sciences, College of Veterinary Medicine, Cornell University（康奈尔大学兽医学院人口医学与诊断科学系）；Department of Radiology, The First Affiliated Hospital of Jinan University（暨南大学第一附属医院放射科）；Department of Radiology, Stanford University School of Medicine（斯坦福大学医学院放射科）；Department of Computer Science and Engineering, The Chinese University of Hong Kong（香港中文大学计算机科学与工程系）；Department of Biomedical Informatics, Harvard Medical School（哈佛医学院生物医学信息学系）
University of Technology Sydney（悉尼科技大学）；IBM Australia Limited（IBM澳大利亚有限公司）；Sejong University（世宗大学）；Universiti Kebangsaan Malaysia（马来西亚国民大学）；Charles Sturt University（查尔斯特大学）
Lung cancer is the leading cause of cancer death and morbidity worldwide. Many studies have shown machine learning models to be effective at detecting lung nodules from chest X-ray images. However, these techniques have yet to be embraced by the medical community due to several practical, ethical, and regulatory constraints stemming from the black-box nature of deep learning models. Additionally, most lung nodules visible on chest X-ray are benign; therefore, the narrow task of computer vision-based lung nodule detection cannot be equated to automated lung cancer detection. Addressing both con…
University of Toronto（多伦多大学）；Vector Institute（向量研究所）；Massachusetts Institute of Technology（麻省理工学院）
Machine learning systems have received much attention recently for their ability to achieve expert-level performance on clinical tasks, particularly in medical imaging. Here, we examine the extent to which state-of-the-art deep learning classifiers trained to yield diagnostic labels from X-ray images are biased with respect to protected attributes. We train convolution neural networks to predict 14 diagnostic labels in 3 prominent public chest X-ray datasets: MIMIC-CXR, Chest-Xray8, CheXpert, as well as a multi-site aggregation of all those datasets. We evaluate the TPR disparity -- the differ…
A data- and compute-efficient chest X-ray foundation model beyond aggressive scaling
Stanford Center for Artificial Intelligence in Medicine and Imaging（斯坦福大学人工智能医学与成像中心）；Stanford University（斯坦福大学）；Department of Radiology（放射科）；Department of Computer Science（计算机科学系）；Department of Pediatrics（儿科）；Department of Biomedical Data Science（生物医学数据科学系）；Department of Medicine（医学系）
University of California, San Diego（加利福尼亚大学圣地亚哥分校）；Google（谷歌）
Automatic medical image report generation has drawn growing attention due to its potential to alleviate radiologists' workload. Existing work on report generation often trains encoder-decoder networks to generate complete reports. However, such models are affected by data bias (e.g.~label imbalance) and face common issues inherent in text generation models (e.g.~repetition). In this work, we focus on reporting abnormal findings on radiology images; instead of training on complete radiology reports, we propose a method to identify abnormal findings from the reports in addition to grouping them …
University of Technology Sydney（悉尼科技大学）；Sun Yat-Sen University（中山大学）；Peng Cheng National Lab（鹏城国家实验室）；The University of Hong Kong（香港大学）；Mohamed bin Zayed University of Artificial Intelligence（穆罕默德·本·扎耶德人工智能大学）
Automatic radiology reporting has great clinical potential to relieve radiologists from heavy workloads and improve diagnosis interpretation. Recently, researchers have enhanced data-driven neural networks with medical knowledge graphs to eliminate the severe visual and textual bias in this task. The structures of such graphs are exploited by using the clinical dependencies formed by the disease topic tags via general knowledge and usually do not update during the training process. Consequently, the fixed graphs can not guarantee the most appropriate scope of knowledge and limit the effectiven…
University College London（伦敦大学学院）；King’s College London（伦敦国王学院）；Royal Free London NHS Foundation Trust（伦敦皇家自由NHS基金会信托）
During the COVID-19 pandemic, the sheer volume of imaging performed in an emergency setting for COVID-19 diagnosis has resulted in a wide variability of clinical CXR acquisitions. This variation is seen in the CXR projections used, image annotations added and in the inspiratory effort and degree of rotation of clinical images. The image analysis community has attempted to ease the burden on overstretched radiology departments during the pandemic by developing automated COVID-19 diagnostic algorithms, the input for which has been CXR imaging. Large publicly available CXR datasets have been leve…
Purpose: The purpose of this study was to observe change in accuracies of convolutional neural networks (CNN) models (ratio of correct classifications to total predictions) on thoracic radiological images by creating different binary classification models based on age, gender, and image pre-processing filters on 14 pathologies. Methodology: This is a quantitative research exploring variation in CNN model accuracies. Radiological thoracic images were divided by age and gender and pre-processed by various image processing filters. Findings: We found partial support for enhancement to model accur…
Grounded Knowledge-Enhanced Medical Vision-Language Pre-training for Chest X-Ray
The Chinese University of Hong Kong（香港中文大学）；CU Lab for AI in Radiology (CLAIR), The Chinese University of Hong Kong（香港中文大学放射学人工智能实验室（CLAIR））；Shanghai Jiao Tong University（上海交通大学）；Shanghai AI Laboratory（上海人工智能实验室）；China Unicom Global Limited（中国联通国际有限公司）
Looking Beyond What You See: An Empirical Analysis on Subgroup Intersectional Fairness for Multi-label Chest X-ray Classification Using Social Determinants of Racial Health Inequities
Massachusetts Institute of Technology（麻省理工学院）；University at Buffalo（布法罗大学）
University of Leeds（利兹大学）；Leeds Teaching Hospitals NHS Trust（利兹教学医院 NHS 信托基金）
Since the emergence of COVID-19, deep learning models have been developed to identify COVID-19 from chest X-rays. With little to no direct access to hospital data, the AI community relies heavily on public data comprising numerous data sources. Model performance results have been exceptional when training and testing on open-source data, surpassing the reported capabilities of AI in pneumonia-detection prior to the COVID-19 outbreak. In this study impactful models are trained on a widely used open-source data and tested on an external test set and a hospital dataset, for the task of classifyin…
DeBoneDiT: Depth-Driven Conditional Bridge Diffusion Transformers for Bone Suppression in Chest X-Ray Images
Zhejiang University（浙江大学）；Hangzhou Dianzi University（杭州电子科技大学）；City University of Hong Kong（香港城市大学）；National Cancer Center/National Clinical Research Center for Cancer/Cancer Hospital & Shenzhen Hospital, Chinese Academy of Medical Sciences and Peking Union Medical College（国家癌症中心/国家临床医学研究中心/中国医学科学院北京协和医学院肿瘤医院深圳医院）；Shenzhen Research Institute of Big Data（深圳大数据研究院）
Hybrid Explanation-Guided Learning for Transformer-Based Chest X-Ray Diagnosis
ARTORG Center for Biomedical Engineering Research, University of Bern（ARTORG生物医学工程研究中心，伯恩大学）；Inselspital (Bern University Hospital)（Inselspital（伯恩大学医院））；Insel Gruppe Bern Universitätsinstitut für Diagnostische, Interventionelle und Pädiatrische Radiologie（Bern大学诊断、介入和儿科放射学研究所）；Department of Radiation Oncology, Inselspital, Bern University Hospital（放射肿瘤科，Inselspital，伯恩大学医院）
University of Turin（都灵大学）；Azienda Ospedaliera Città della Salute e della Scienza（健康与科学城医院）
The possibility to use widespread and simple chest X-ray (CXR) imaging for early screening of COVID-19 patients is attracting much interest from both the clinical and the AI community. In this study we provide insights and also raise warnings on what is reasonable to expect by applying deep-learning to COVID classification of CXR images. We provide a methodological guide and critical reading of an extensive set of statistical results that can be obtained using currently available datasets. In particular, we take the challenge posed by current small size COVID data and show how significant can …
The reliability of machine learning models can be compromised when trained on low quality data. Many large-scale medical imaging datasets contain low quality labels extracted from sources such as medical reports. Moreover, images within a dataset may have heterogeneous quality due to artifacts and biases arising from equipment or measurement errors. Therefore, algorithms that can automatically identify low quality data are highly desired. In this study, we used data Shapley, a data valuation metric, to quantify the value of training data to the performance of a pneumonia detection algorithm in…
Comparing radiologists' gaze and saliency maps generated by interpretability methods for chest x-rays
Scientific Computing and Imaging Institute, University of Utah（犹他大学科学计算与成像研究所）；School of Computing, University of Utah（犹他大学计算学院）；Department of Psychology, University of Utah（犹他大学心理学系）；Department of Radiology and Imaging Sciences, University of Utah（犹他大学放射学与影像科学系）
University of Minnesota（明尼苏达大学）；Emory University（埃默里大学）；M Health Fairview Informatics（M Health Fairview 信息学）；Indiana University（印第安纳大学）；North Memorial Health Hospital（North Memorial Health 医院）
Importance: An artificial intelligence (AI)-based model to predict COVID-19 likelihood from chest x-ray (CXR) findings can serve as an important adjunct to accelerate immediate clinical decision making and improve clinical decision making. Despite significant efforts, many limitations and biases exist in previously developed AI diagnostic models for COVID-19. Utilizing a large set of local and international CXR images, we developed an AI model with high performance on temporal and external validation. Conclusions and Relevance: AI-based diagnostic tools may serve as an adjunct, but not replace…
National Library of Medicine, National Institutes of Health（美国国家医学图书馆，美国国立卫生研究院）
Medical images commonly exhibit multiple abnormalities. Predicting them requires multi-class classifiers whose training and desired reliable performance can be affected by a combination of factors, such as, dataset size, data source, distribution, and the loss function used to train the deep neural networks. Currently, the cross-entropy loss remains the de-facto loss function for training deep learning classifiers. This loss function, however, asserts equal learning from all classes, leading to a bias toward the majority class. In this work, we benchmark various state-of-the-art loss functions…
Technical University of Munich (TUM)（慕尼黑工业大学）；Klinikum Rechts der Isar（伊萨尔河畔大学医院）；Imperial College London（伦敦帝国理工学院）
In medical imaging, un-, semi-, or self-supervised pathology detection is often approached with anomaly- or out-of-distribution detection methods, whose inductive biases are not intentionally directed towards detecting pathologies, and are therefore sub-optimal for this task. To tackle this problem, we propose AutoSeg, an engine that can generate diverse artificial anomalies that resemble the properties of real-world pathologies. Our method can accurately segment unseen artificial anomalies and outperforms existing methods for pathology detection on a challenging real-world dataset of Chest X-…
University of Michigan（密歇根大学）；Institute for Healthcare Policy & Innovation（医疗政策与创新研究所）；Department of Electrical Engineering and Computer Science（电气工程与计算机科学系）；Department of Radiology（放射科）；Department of Internal Medicine（内科）
While deep learning has shown promise in improving the automated diagnosis of disease based on chest X-rays, deep networks may exhibit undesirable behavior related to shortcuts. This paper studies the case of spurious class skew in which patients with a particular attribute are spuriously more likely to have the outcome of interest. For instance, clinical protocols might lead to a dataset in which patients with pacemakers are disproportionately likely to have congestive heart failure. This skew can lead to models that take shortcuts by heavily relying on the biased attribute. We explore this p…
