Recently, paired (e.g. Pix2pix) and unpaired (e.g. CycleGAN) image-to-image translation methods have shown effective in medical imaging tasks. In practice, however, it can be difficult to apply these deep models on medical data volumes, such as from MR and CT scans, since such data volumes tend to be 3-dimensional and of a high spatial resolution pushing the limits of the memory constraints of GPU hardware that is typically used to train these models. Recent studies in the field of invertible neural networks have shown that reversible neural networks do not require to store intermediate activations for training. We use the RevGAN model that makes use of this property to perform memory-efficient partially-reversible image-to-image translation. We demonstrate this by performing a 3D super-resolution and a 3D domain-adaptation task on chest CT data volumes.
Institute of Medical Science, University of Toronto（多伦多大学医学科学研究所）；The Hospital for Sick Children（病童医院）；University Health Network（大学健康网络）；Department of Mechanical and Industrial Engineering, University of Toronto（多伦多大学机械与工业工程系）
Successful training of convolutional neural networks (CNNs) requires a substantial amount of data. With small datasets networks generalize poorly. Data Augmentation techniques improve the generalizability of neural networks by using existing training data more effectively. Standard data augmentation methods, however, produce limited plausible alternative data. Generative Adversarial Networks (GANs) have been utilized to generate new data and improve the performance of CNNs. Nevertheless, data augmentation techniques for training GANs are under-explored compared to CNNs. In this work, we propos…
The traditional reversible data hiding technique is based on cover image modification which inevitably leaves some traces of rewriting that can be more easily analyzed and attacked by the warder. Inspired by the cover synthesis steganography based generative adversarial networks, in this paper, a novel generative reversible data hiding scheme (GRDH) by image translation is proposed. First, an image generator is used to obtain a realistic image, which is used as an input to the image-to-image translation model with CycleGAN. After image translation, a stego image with different semantic informa…
University of Florence（佛罗伦萨大学）；University of Siena（锡耶纳大学）
Multi-organ segmentation of X-ray images is of fundamental importance for computer aided diagnosis systems. However, the most advanced semantic segmentation methods rely on deep learning and require a huge amount of labeled images, which are rarely available due to both the high cost of human resources and the time required for labeling. In this paper, we present a novel multi-stage generation algorithm based on Generative Adversarial Networks (GANs) that can produce synthetic images along with their semantic labels and can be used for data augmentation. The main feature of the method is that,…
University of the Witwatersrand, Johannesburg（金山大学（约翰内斯堡））；Helen Joseph Hospital（海伦·约瑟夫医院）
Chest x-rays are a vital tool in the workup of many patients. Similar to most medical imaging modalities, they are profoundly multi-modal and are capable of visualising a variety of combinations of conditions. There is an ever pressing need for greater quantities of labelled data to develop new diagnostic tools, however this is in direct opposition to concerns regarding patient confidentiality which constrains access through permission requests and ethics approvals. Previous work has sought to address these concerns by creating class-specific GANs that synthesise images to augment training dat…
A common problem in computer vision -- particularly in medical applications -- is a lack of sufficiently diverse, large sets of training data. These datasets often suffer from severe class imbalance. As a result, networks often overfit and are unable to generalize to novel examples. Generative Adversarial Networks (GANs) offer a novel method of synthetic data augmentation. In this work, we evaluate the use of GAN- based data augmentation to artificially expand the CheXpert dataset of chest radiographs. We compare performance to traditional augmentation and find that GAN-based augmentation lead…
Caring Research India（Caring Research 印度）；Indian Institute of Science（印度科学理工学院）；Pupil Labs；Ankara University（安卡拉大学）
Disease-aware image editing by means of generative adversarial networks (GANs) constitutes a promising avenue for advancing the use of AI in the healthcare sector. Here, we present a proof of concept of this idea. While GAN-based techniques have been successful in generating and manipulating natural images, their application to the medical domain, however, is still in its infancy. Working with the CheXpert data set, we show that StyleGAN can be trained to generate realistic chest X-rays. Inspired by the Cyclic Reverse Generator (CRG) framework, we train an encoder that allows for faithfully in…
SARS-CoV-2, also known as COVID-19 or Coronavirus, is a viral contagious disease that is infected by a novel coronavirus, and has been rapidly spreading across the globe. It is very important to test and isolate people to reduce spread, and from here comes the need to do this quickly and efficiently. According to some studies, Chest-CT outperforms RT-PCR lab testing, which is the current standard, when diagnosing COVID-19 patients. Due to this, computer vision researchers have developed various deep learning systems that can predict COVID-19 using a Chest-CT scan correctly to a certain degree.…
Recent works show that Generative Adversarial Networks (GANs) can be successfully applied to chest X-ray data augmentation for lung disease recognition. However, the implausible and distorted pathology features generated from the less than perfect generator may lead to wrong clinical decisions. Why not keep the original pathology region? We proposed a novel approach that allows our generative model to generate high quality plausible images that contain undistorted pathology areas. The main idea is to design a training scheme based on an image-to-image translation network to introduce variation…
Imaging Biomarkers and Computer-aided Diagnosis Laboratory, Radiology and Imaging Sciences, National Institutes of Health Clinical Center（美国国立卫生研究院临床中心放射学与成像科学部成像生物标志物和计算机辅助诊断实验室）；Ping An Technology Co., Ltd.（平安科技有限公司）
Dual-energy (DE) chest radiography provides the capability of selectively imaging two clinically relevant materials, namely soft tissues, and osseous structures, to better characterize a wide variety of thoracic pathology and potentially improve diagnosis in posteroanterior (PA) chest radiographs. However, DE imaging requires specialized hardware and a higher radiation dose than conventional radiography, and motion artifacts sometimes happen due to involuntary patient motion. In this work, we learn the mapping between conventional radiographs and bone suppressed radiographs. Specifically, we p…
This review presents a comprehensive study on the role of GANs in addressing the challenges related to COVID-19 data scarcity and diagnosis. It is the first review that summarizes the different GANs methods and the lungs images datasets for COVID-19. It attempts to answer the questions related to applications of GANs, popular GAN architectures, frequently used image modalities, and the availability of source code. This review included 57 full-text studies that reported the use of GANs for different applications in COVID-19 lungs images data. Most of the studies (n=42) used GANs for data augmen…
COMSATS University Islamabad（COMSATS伊斯兰堡大学）；National University of Ireland, Galway（爱尔兰国立高威大学）；Umeå University（于默奥大学）
Chest X-ray (CXR) is a low-cost medical imaging technique. It is a common procedure for the identification of many respiratory diseases compared to MRI, CT, and PET scans. This paper presents the use of generative adversarial networks (GAN) to perform the task of lung segmentation on a given CXR. GANs are popular to generate realistic data by learning the mapping from one domain to another. In our work, the generator of the GAN is trained to generate a segmented mask of a given input CXR. The discriminator distinguishes between a ground truth and the generated mask, and updates the generator t…
Stanford University（斯坦福大学）；Mila, Quebec AI Institute（魁北克人工智能研究所米拉）；Nuance Communications（纽昂斯通讯公司）
Motivation: Traditional image attribution methods struggle to satisfactorily explain predictions of neural networks. Prediction explanation is important, especially in medical imaging, for avoiding the unintended consequences of deploying AI systems when false positive predictions can impact patient care. Thus, there is a pressing need to develop improved models for model explainability and introspection. Specific problem: A new approach is to transform input images to increase or decrease features which cause the prediction. However, current approaches are difficult to implement as they are m…
The coronavirus disease (COVID-19) has resulted in a pandemic crippling the a breadth of services critical to daily life. Segmentation of lung infections in computerized tomography (CT) slices could be be used to improve diagnosis and understanding of COVID-19 in patients. Deep learning systems lack interpretability because of their black box nature. Inspired by human communication of complex ideas through language, we propose a symbolic framework based on emergent languages for the segmentation of COVID-19 infections in CT scans of lungs. We model the cooperation between two artificial agents…
Intelligent Medical Image Computing Systems (IMICS) Lab（智能医学影像计算系统实验室）；The Hospital for Sick Children（病童医院）；University of Toronto（多伦多大学）
COVID-19 spread across the globe at an immense rate has left healthcare systems incapacitated to diagnose and test patients at the needed rate. Studies have shown promising results for detection of COVID-19 from viral bacterial pneumonia in chest X-rays. Automation of COVID-19 testing using medical images can speed up the testing process of patients where health care systems lack sufficient numbers of the reverse-transcription polymerase chain reaction (RT-PCR) tests. Supervised deep learning models such as convolutional neural networks (CNN) need enough labeled data for all classes to correct…
Comparative clinical evaluation of "memory-efficient" synthetic 3d generative adversarial networks (gan) head-to-head to state of art: results on computed tomography of the chest
Politecnico di Milano（米兰理工大学）；Fondazione IRCCS Policlinico San Matteo Pavia（圣马泰奥基金会IRCCS帕维亚医院）；Università degli studi di Pavia（帕维亚大学）；IULM University（IULM大学）
Inception Institute of Artificial Intelligence（人工智能 inception 研究所）
Medical image registration is an important task in automated analysis of multi-modal images and temporal data involving multiple patient visits. Conventional approaches, although useful for different image types, are time consuming. Of late, deep learning (DL) based image registration methods have been proposed that outperform traditional methods in terms of accuracy and time. However,DL based methods are heavily dependent on training data and do not generalize well when presented with images of different scanners or anatomies. We present a DL based approach that can perform medical image regi…
University of Cambridge（剑桥大学）；Royal Brompton Hospital（皇家布朗普顿医院）；Imperial College London（帝国理工学院）
Single image super-resolution (SISR) aims to obtain a high-resolution output from one low-resolution image. Currently, deep learning-based SISR approaches have been widely discussed in medical image processing, because of their potential to achieve high-quality, high spatial resolution images without the cost of additional scans. However, most existing methods are designed for scale-specific SR tasks and are unable to generalise over magnification scales. In this paper, we propose an approach for medical image arbitrary-scale super-resolution (MIASSR), in which we couple meta-learning with gen…
Imaging technology center, Fujifilm corporation（富士胶片公司成像技术中心）；Center for Artificial Intelligence Research, University of Tsukuba（筑波大学人工智能研究中心）；Department of computer science and engineering, Waseda university（早稻田大学计算机科学与工程系）
Many CT slice images are stored with large slice intervals to reduce storage size in clinical practice. This leads to low resolution perpendicular to the slice images (i.e., z-axis), which is insufficient for 3D visualization or image analysis. In this paper, we present a novel architecture based on conditional Generative Adversarial Networks (cGANs) with the goal of generating high resolution images of main body parts including head, chest, abdomen and legs. However, GANs are known to have a difficulty with generating a diversity of patterns due to a phenomena known as mode collapse. To overc…
ETH Zurich（苏黎世联邦理工学院）；Max Planck Institute for Intelligent Systems（马克斯·普朗克智能系统研究所）；University Hospital of Tübingen（蒂宾根大学医院）；Kantonsspital Winterthur（温特图尔州立医院）；GlaxoSmithKline（葛兰素史克）
Privacy concerns around sharing personally identifiable information are a major practical barrier to data sharing in medical research. However, in many cases, researchers have no interest in a particular individual's information but rather aim to derive insights at the level of cohorts. Here, we utilize Generative Adversarial Networks (GANs) to create derived medical imaging datasets consisting entirely of synthetic patient data. The synthetic images ideally have, in aggregate, similar statistical properties to those of a source dataset but do not contain sensitive personal information. We ass…
University of Maryland, Baltimore County（马里兰大学巴尔的摩县分校）；National Institutes of Health Clinical Center（美国国立卫生研究院临床中心）；Networking Health（网络健康）
COVID-19 is a novel infectious disease responsible for over 800K deaths worldwide as of August 2020. The need for rapid testing is a high priority and alternative testing strategies including X-ray image classification are a promising area of research. However, at present, public datasets for COVID19 x-ray images have low data volumes, making it challenging to develop accurate image classifiers. Several recent papers have made use of Generative Adversarial Networks (GANs) in order to increase the training data volumes. But realistic synthetic COVID19 X-rays remain challenging to generate. We p…
The University of Texas MD Anderson Cancer Center（德克萨斯大学MD安德森癌症中心）；Rice University（莱斯大学）；University of California San Diego（加州大学圣地亚哥分校）
Although generative adversarial networks (GANs) have shown promise in medical imaging, they have four main limitations that impeded their utility: computational cost, data requirements, reliable evaluation measures, and training complexity. Our work investigates each of these obstacles in a novel application of StyleGAN2-ADA to high-resolution medical imaging datasets. Our dataset is comprised of liver-containing axial slices from non-contrast and contrast-enhanced computed tomography (CT) scans. Additionally, we utilized four public datasets composed of various imaging modalities. We trained …
