Encoding CT Anatomy Knowledge for Unpaired Chest X-ray Image Decomposition
Imperial College London（帝国理工学院）；University of Chinese Academy of Sciences（中国科学院大学）；Institute of Computing Technology, CAS（中国科学院计算技术研究所）；Peng Cheng Laboratory（鹏城实验室）；Southest University（东南大学）；Nankai University（南开大学）
Zeju Li, Han Li, Hu Han, Gonglei Shi, Jiannan Wang, S. Kevin Zhou
Although chest X-ray (CXR) offers a 2D projection with overlapped anatomies, it is widely used for clinical diagnosis. There is clinical evidence supporting that decomposing an X-ray image into different components (e.g., bone, lung and soft tissue) improves diagnostic value. We hereby propose a decomposition generative adversarial network (DecGAN) to anatomically decompose a CXR image but with unpaired data. We leverage the anatomy knowledge embedded in CT, which features a 3D volume with clearly visible anatomies. Our key idea is to embed CT priori decomposition knowledge into the latent space of unpaired CXR autoencoder. Specifically, we train DecGAN with a decomposition loss, adversarial losses, cycle-consistency losses and a mask loss to guarantee that the decomposed results of the latent space preserve realistic body structures. Extensive experiments demonstrate that DecGAN provides superior unsupervised CXR bone suppression results and the feasibility of modulating CXR components by latent space disentanglement. Furthermore, we illustrate the diagnostic value of DecGAN and demonstrate that it outperforms the state-of-the-art approaches in terms of predicting 11 out of 14 common lung diseases.
Concordia Institute for Information Systems Engineering（康考迪亚大学信息系统工程研究所）；Concordia University（康考迪亚大学）
Motivated by the lack of publicly available datasets of chest radiographs of positive patients with Coronavirus disease 2019 (COVID-19), we build the first-of-its-kind open dataset of synthetic COVID-19 chest X-ray images of high fidelity using an unsupervised domain adaptation approach by leveraging class conditioning and adversarial training. Our contributions are twofold. First, we show considerable performance improvements on COVID-19 detection using various deep learning architectures when employing synthetic images as additional training set. Second, we show how our image synthesis metho…
Anatomy-preserving unpaired cone-beam CT refinement for image-guided radiotherapy using pseudo-label guided diffusion
Shenzhen Institutes of Advanced Technology, Chinese Academy of Sciences（中国科学院深圳先进技术研究院）；Southern University of Science and Technology（南方科技大学）；Huashan Hospital, Fudan University（复旦大学附属华山医院）
Imaging Biomarkers and Computer-aided Diagnosis Laboratory, Radiology and Imaging Sciences, National Institutes of Health Clinical Center（美国国立卫生研究院临床中心放射学与成像科学部成像生物标志物和计算机辅助诊断实验室）；Ping An Technology Co., Ltd.（平安科技有限公司）
Dual-energy (DE) chest radiography provides the capability of selectively imaging two clinically relevant materials, namely soft tissues, and osseous structures, to better characterize a wide variety of thoracic pathology and potentially improve diagnosis in posteroanterior (PA) chest radiographs. However, DE imaging requires specialized hardware and a higher radiation dose than conventional radiography, and motion artifacts sometimes happen due to involuntary patient motion. In this work, we learn the mapping between conventional radiographs and bone suppressed radiographs. Specifically, we p…
AREPAS: Anomaly Detection in Fine-Grained Anatomy with Reconstruction-Based Semantic Patch-Scoring
Computational Imaging Research Lab, Department of Biomedical Imaging and Image-guided Therapy, Medical University of Vienna（医学影像与影像引导治疗系计算成像研究实验室，维也纳医学大学）；Comprehensive Center for Artificial Intelligence in Medicine, Medical University of Vienna（医学人工智能综合中心，维也纳医学大学）
National Institutes of Health（美国国立卫生研究院）；Ping An Technology Co., Ltd.（平安科技有限公司）
In this work, we exploit the unsupervised domain adaptation problem for radiology image interpretation across domains. Specifically, we study how to adapt the disease recognition model from a labeled source domain to an unlabeled target domain, so as to reduce the effort of labeling each new dataset. To address the shortcoming of cross-domain, unpaired image-to-image translation methods which typically ignore class-specific semantics, we propose a task-driven, discriminatively trained, cycle-consistent generative adversarial network, termed TUNA-Net. It is able to preserve 1) low-level details…
Bangladesh University of Engineering and Technology（孟加拉国工程技术大学）；Georgia Institute of Technology（佐治亚理工学院）
Thoracic disease detection from chest radiographs using deep learning methods has been an active area of research in the last decade. Most previous methods attempt to focus on the diseased organs of the image by identifying spatial regions responsible for significant contributions to the model's prediction. In contrast, expert radiologists first locate the prominent anatomical structures before determining if those regions are anomalous. Therefore, integrating anatomical knowledge within deep learning models could bring substantial improvement in automatic disease classification. Motivated by …
University of Maryland（马里兰大学）；PAII Inc.（PAII公司）；Alibaba Group DAMO Academy（阿里巴巴集团达摩院）；Xpeng Motors（小鹏汽车）；Ping An Technology（平安科技）；Chang Gung Memorial Hospital（长庚纪念医院）
Osteoporosis is a common chronic metabolic bone disease often under-diagnosed and under-treated due to the limited access to bone mineral density (BMD) examinations, e.g. via Dual-energy X-ray Absorptiometry (DXA). This paper proposes a method to predict BMD from Chest X-ray (CXR), one of the most commonly accessible and low-cost medical imaging examinations. Our method first automatically detects Regions of Interest (ROIs) of local CXR bone structures. Then a multi-ROI deep model with transformer encoder is developed to exploit both local and global information in the chest X-ray image for ac…
Z 2 Sky Technologies Inc.（Z 2 天工科技公司）；University of Rochester（罗切斯特大学）；Key Lab of Intelligent Information Processing of Chinese Academy of Sciences (CAS), Institute of Computing Technology, CAS（中国科学院计算技术研究所 中国科学院智能信息处理重点实验室）；University of Science and Technology of China（中国科学技术大学）；Suzhou Institute for Advance Research, University of Science and Technology of China（中国科学技术大学苏州高等研究院）
Spinal surgery planning necessitates automatic segmentation of vertebrae in cone-beam computed tomography (CBCT), an intraoperative imaging modality that is widely used in intervention. However, CBCT images are of low-quality and artifact-laden due to noise, poor tissue contrast, and the presence of metallic objects, causing vertebra segmentation, even manually, a demanding task. In contrast, there exists a wealth of artifact-free, high quality CT images with vertebra annotations. This motivates us to build a CBCT vertebra segmentation model using unpaired CT images with annotations. To overco…
Supervised Pix2Pix and unsupervised Cycle-consistency are two modes that dominate the field of medical image-to-image translation. However, neither modes are ideal. The Pix2Pix mode has excellent performance. But it requires paired and well pixel-wise aligned images, which may not always be achievable due to respiratory motion or anatomy change between times that paired images are acquired. The Cycle-consistency mode is less stringent with training data and works well on unpaired or misaligned images. But its performance may not be optimal. In order to break the dilemma of the existing modes, …
Fetal brain magnetic resonance imaging (MRI) offers exquisite images of the developing brain but is not suitable for anomaly screening. For this ultrasound (US) is employed. While expert sonographers are adept at reading US images, MR images are much easier for non-experts to interpret. Hence in this paper we seek to produce images with MRI-like appearance directly from clinical US images. Our own clinical motivation is to seek a way to communicate US findings to patients or clinical professionals unfamiliar with US, but in medical image analysis such a capability is potentially useful, for in…
VoxelCloud Inc（体素科技公司）；University of California, Los Angeles（加利福尼亚大学洛杉矶分校）；Shanghai Jiao Tong University（上海交通大学）
Deep learning models tend to underperform in the presence of domain shifts. Domain transfer has recently emerged as a promising approach wherein images exhibiting a domain shift are transformed into other domains for augmentation or adaptation. However, with the absence of paired and annotated images, models merely learned by adversarial loss and cycle consistency loss could result in poor consistency of anatomy structures during the translation. Additionally, the complexity of learning multi-domain transfer could significantly increase with the number of target domains and source images. In t…
Structure-Preserving Synthesis: MaskGAN for Unpaired MR-CT Translation
Australian Institute for Machine Learning, University of Adelaide（阿德莱德大学澳大利亚机器学习研究所）；Flinders Health and Medical Research Institute, Flinders University（弗林德斯大学弗林德斯健康与医学研究所）
Radboud University Medical Center（拉德堡德大学医学中心）；Netherlands Cancer Institute (NKI)（荷兰癌症研究所）；Z 2 Sky Technologies Inc.（Z 2 Sky 技术公司）；Johns Hopkins University（约翰斯·霍普金斯大学）；University of Science and Technology of China（中国科学技术大学）；Institute of Computing Technology, CAS（中国科学院计算技术研究所）
Clinical evidence has shown that rib-suppressed chest X-rays (CXRs) can improve the reliability of pulmonary disease diagnosis. However, previous approaches on generating rib-suppressed CXR face challenges in preserving details and eliminating rib residues. We hereby propose a GAN-based disentanglement learning framework called Rib Suppression GAN, or RSGAN, to perform rib suppression by utilizing the anatomical knowledge embedded in unpaired computed tomography (CT) images. In this approach, we employ a residual map to characterize the intensity difference between CXR and the corresponding ri…
Recently, paired (e.g. Pix2pix) and unpaired (e.g. CycleGAN) image-to-image translation methods have shown effective in medical imaging tasks. In practice, however, it can be difficult to apply these deep models on medical data volumes, such as from MR and CT scans, since such data volumes tend to be 3-dimensional and of a high spatial resolution pushing the limits of the memory constraints of GPU hardware that is typically used to train these models. Recent studies in the field of invertible neural networks have shown that reversible neural networks do not require to store intermediate activa…
In this work, we propose a method to reject out-of-distribution samples which can be adapted to any network architecture and requires no additional training data. Publicly available chest x-ray data (38,353 images) is used to train a standard ResNet-50 model to detect emphysema. Feature activations of intermediate layers are used as descriptors defining the training data distribution. A novel metric, FRODO, is measured by using the Mahalanobis distance of a new test sample to the training data distribution. The method is tested using a held-out test dataset of 21,176 chest x-rays (in-distribut…
ContourDiff: Unpaired Medical Image Translation with Structural Consistency
Department of Electrical and Computer Engineering, Duke University（杜克大学电气与计算机工程系）；Department of Biostatistics & Bioinformatics, Duke University（杜克大学生物统计学与生物信息学系）；Department of Radiology, Duke University（杜克大学放射学系）；Department of Computer Science, Duke University（杜克大学计算机科学系）
Siemens Healthineers（西门子医疗）；University of Michigan（密歇根大学）；Massachusetts General Hospital（麻省总医院）；Harvard Medical School（哈佛医学院）
The interpretation of medical images is a challenging task, often complicated by the presence of artifacts, occlusions, limited contrast and more. Most notable is the case of chest radiography, where there is a high inter-rater variability in the detection and classification of abnormalities. This is largely due to inconclusive evidence in the data or subjective definitions of disease appearance. An additional example is the classification of anatomical views based on 2D Ultrasound images. Often, the anatomical context captured in a frame is not sufficient to recognize the underlying anatomy. …
