Dual-energy (DE) chest radiographs provide greater diagnostic information than standard radiographs by separating the image into bone and soft tissue, revealing suspicious lesions which may otherwise be obstructed from view. However, acquisition of DE images requires two physical scans, necessitating specialized hardware and processing, and images are prone to motion artifact. Generation of virtual DE images from standard, single-shot chest radiographs would expand the diagnostic value of standard radiographs without changing the acquisition procedure. We present a Multi-scale Conditional Adversarial Network (MCA-Net) which produces high-resolution virtual DE bone images from standard, single-shot chest radiographs. Our proposed MCA-Net is trained using the adversarial network so that it learns sharp details for the production of high-quality bone images. Then, the virtual DE soft tissue image is generated by processing the standard radiograph with the virtual bone image using a cross projection transformation. Experimental results from 210 patient DE chest radiographs demonstrated that the algorithm can produce high-quality virtual DE chest radiographs. Important structures were preserved, such as coronary calcium in bone images and lung lesions in soft tissue images. The average structure similarity index and the peak signal to noise ratio of the produced bone images in testing data were 96.4 and 41.5, which are significantly better than results from previous methods. Furthermore, our clinical evaluation results performed on the publicly available dataset indicates the clinical values of our algorithms. Thus, our algorithm can produce high-quality DE images that are potentially useful for radiologists, computer-aided diagnostics, and other diagnostic tasks.
National Institutes of Health, Clinical Center（美国国立卫生研究院临床中心）；Ping An Technology Co., Ltd.（平安科技有限公司）
In this work, we exploit the task of joint classification and weakly supervised localization of thoracic diseases from chest radiographs, with only image-level disease labels coupled with disease severity-level (DSL) information of a subset. A convolutional neural network (CNN) based attention-guided curriculum learning (AGCL) framework is presented, which leverages the severity-level attributes mined from radiology reports. Images in order of difficulty (grouped by different severity-levels) are fed to CNN to boost the learning gradually. In addition, highly confident samples (measured by cla…
ViT3Flow: A Test-Time Training Transformer MeanFlow for Postoperative Radiograph Synthesis in Scoliosis
The University of Hong Kong（香港大学）；The University of Hong Kong-Shenzhen Hospital（香港大学深圳医院）；The Hong Kong University of Science and Technology (Guangzhou)（香港科技大学（广州））；The Hong Kong University of Science and Technology（香港科技大学）；The Chinese University of Hong Kong（香港中文大学）；Qilu Hospital of Shandong University（山东大学齐鲁医院）
Currently, Segmentation of bitewing radiograpy images is a very challenging task. The focus of the study is to segment it into caries, enamel, dentin, pulp, crowns, restoration and root canal treatments. The main method of semantic segmentation of bitewing radiograpy images at this stage is the U-shaped deep convolution neural network, but its accuracy is low. in order to improve the accuracy of semantic segmentation of bitewing radiograpy images, this paper proposes the use of Conditional Generative Adversarial network (cGAN) combined with U-shaped network structure (U-Net) approach to semant…
National University of Singapore（新加坡国立大学）；Institute for Infocomm Research, A*STAR（新加坡科技研究局资讯通信研究院）；Singapore University of Technology and Design（新加坡科技设计大学）
Deep learning approaches achieve state-of-the-art performance for classifying radiology images, but rely on large labelled datasets that require resource-intensive annotation by specialists. Both semi-supervised learning and active learning can be utilised to mitigate this annotation burden. However, there is limited work on combining the advantages of semi-supervised and active learning approaches for multi-label medical image classification. Here, we introduce a novel Consistency-based Semi-supervised Evidential Active Learning framework (CSEAL). Specifically, we leverage predictive uncertai…
Florida International University（佛罗里达国际大学）；Technical University of Munich（慕尼黑工业大学）；Stanford University（斯坦福大学）
Chest X-ray radiography is one of the earliest medical imaging technologies and remains one of the most widely-used for diagnosis, screening, and treatment follow up of diseases related to lungs and heart. The literature in this field of research reports many interesting studies dealing with the challenging tasks of bone suppression and organ segmentation but performed separately, limiting any learning that comes with the consolidation of parameters that could optimize both processes. This study, and for the first time, introduces a multitask deep learning model that generates simultaneously t…
Feasibility Study of a Diffusion-Based Model for Cross-Modal Generation of Knee MRI from X-ray: Integrating Radiographic Feature Information
Massachusetts General Hospital（麻省总医院）；Harvard Medical School（哈佛医学院）；University Sorbonne Paris Nord（巴黎北大索邦大学）；German Cancer Research Center（德国癌症研究中心）；Jiangsu Institute of Hematology（江苏血液学研究所）；The First Affiliated Hospital of Soochow University（苏州大学附属第一医院）；Henan University of Chinese Medicine（河南中医药大学）；Athinoula A. Martinos Centre for Biomedical Imaging（阿西努拉·A.马蒂诺斯生物医学成像中心）；IDP Institute（IDP研究所）；University of Orleans（奥尔良大学）
AnyCXR: Human Anatomy Segmentation of Chest X-ray at Any Acquisition Position using Multi-stage Domain Randomized Synthetic Data with Imperfect Annotations and Conditional Joint Annotation Regularization Learning
Department of Radiology, Northwestern University, Chicago, IL, USA（西北大学放射科）；Data Science Institute, Vanderbilt University, Nashville, TN, USA（范德比尔特大学数据科学研究院）；Department of Orthopedics, The Second Hospital of Shanxi Medical University, Taiyuan, Shanxi, P.R. China（山西医科大学第二医院骨科）；Second Clinical Medical College, Shanxi Medical University, Taiyuan, Shanxi, P.R. China（山西医科大学第二临床医学院）；School of Computer Science, University of Sydney, Sydney, NSW, Australia（悉尼大学计算机科学学院）
DeBoneDiT: Depth-Driven Conditional Bridge Diffusion Transformers for Bone Suppression in Chest X-Ray Images
Zhejiang University（浙江大学）；Hangzhou Dianzi University（杭州电子科技大学）；City University of Hong Kong（香港城市大学）；National Cancer Center/National Clinical Research Center for Cancer/Cancer Hospital & Shenzhen Hospital, Chinese Academy of Medical Sciences and Peking Union Medical College（国家癌症中心/国家临床医学研究中心/中国医学科学院北京协和医学院肿瘤医院深圳医院）；Shenzhen Research Institute of Big Data（深圳大数据研究院）
Digitization techniques for biomedical images yield different visual patterns in radiological exams. These differences may hamper the use of data-driven approaches for inference over these images, such as Deep Neural Networks. Another noticeable difficulty in this field is the lack of labeled data, even though in many cases there is an abundance of unlabeled data available. Therefore an important step in improving the generalization capabilities of these methods is to perform Unsupervised and Semi-Supervised Domain Adaptation between different datasets of biomedical images. In order to tackle …
Problem: There is a lack of big data for the training of deep learning models in medicine, characterized by the time cost of data collection and privacy concerns. Generative adversarial networks (GANs) offer both the potential to generate new data, as well as to use this newly generated data, without inclusion of patients' real data, for downstream applications. Approach: A series of GANs were trained and applied for a downstream computer vision spine radiograph abnormality classification task. Separate classifiers were trained with either access or no access to the original imaging. Trained G…
German Cancer Research Center（德国癌症研究中心）；Heidelberg University（海德堡大学）
Image registration is the basis for many applications in the fields of medical image computing and computer assisted interventions. One example is the registration of 2D X-ray images with preoperative three-dimensional computed tomography (CT) images in intraoperative surgical guidance systems. Due to the high safety requirements in medical applications, estimating registration uncertainty is of a crucial importance in such a scenario. However, previously proposed methods, including classical iterative registration methods and deep learning-based methods have one characteristic in common: They…
CAVM: Conditional Autoregressive Vision Model for Contrast-Enhanced Brain Tumor MRI Synthesis
School of Medical Technology, Beijing Institute of Technology（北京理工大学医学技术学院）；School of Integrated Circuits and Electronics, Beijing Institute of Technology（北京理工大学集成电路与电子学院）
University of Toronto（多伦多大学）；Massachusetts Institute of Technology（麻省理工学院）；Vector Institute（矢量研究所）
The automatic generation of radiology reports given medical radiographs has significant potential to operationally and improve clinical patient care. A number of prior works have focused on this problem, employing advanced methods from computer vision and natural language generation to produce readable reports. However, these works often fail to account for the particular nuances of the radiology domain, and, in particular, the critical importance of clinical accuracy in the resulting generated reports. In this work, we present a domain-aware automatic chest X-ray radiology report generation s…
Predictive Enhancement Calibration for Latent Breast MRI Virtual Contrast Enhancement
The First Affiliated Hospital of Chongqing Medical University（重庆医科大学附属第一医院）；Chongqing Municipal Health Commission（重庆市卫生健康委员会）；Chongqing Translational Medicine Center（重庆市转化医学中心）
When trained on multimodal image datasets, normal Generative Adversarial Networks (GANs) are usually outperformed by class-conditional GANs and ensemble GANs, but conditional GANs is restricted to labeled datasets and ensemble GANs lack efficiency. We propose a novel GAN variant called virtual conditional GAN (vcGAN) which is not only an ensemble GAN with multiple generative paths while adding almost zero network parameters, but also a conditional GAN that can be trained on unlabeled datasets without explicit clustering steps or objectives other than the adversary loss. Inside the vcGAN's gene…
Translating Inference-Time Control to Radiology Vision-Language Models: Activation Steering for Pneumonia Classification on Chest X-rays
Universidade Federal de São Paulo (UNIFESP)（圣保罗联邦大学）；Hospital Israelita Albert Einstein（以色列阿尔伯特·爱因斯坦医院）；Stanford University School of Medicine（斯坦福大学医学院）；DASA；University of Texas Southwestern Medical Center (UTSW)（德克萨斯大学西南医学中心）；Eden
Seoul National University（首尔国立大学）；Hankuk University of Foreign Studies（韩国外国语大学）
Suppressing bones on chest X-rays such as ribs and clavicle is often expected to improve pathologies classification. These bones can interfere with a broad range of diagnostic tasks on pulmonary disease except for musculoskeletal system. Current conventional method for acquisition of bone suppressed X-rays is dual energy imaging, which captures two radiographs at a very short interval with different energy levels; however, the patient is exposed to radiation twice and the artifacts arise due to heartbeats between two shots. In this paper, we introduce a deep generative model trained to predict…
Harvard Medical School（哈佛医学院）；Harvard University（哈佛大学）；The University of Adelaide（阿德莱德大学）
Medical students and radiology trainees typically view thousands of images in order to "train their eye" to detect the subtle visual patterns necessary for diagnosis. Nevertheless, infrastructural and legal constraints often make it difficult to access and quickly query an abundance of images with a user-specified feature set. In this paper, we use a conditional generative adversarial network (GAN) to synthesize $1024\times1024$ pixel pelvic radiographs that can be queried with conditioning on fracture status. We demonstrate that the conditional GAN learns features that distinguish fractures f…
Flow-based conditional cardiac anatomy generation for virtual cohorts
Delft University of Technology（代尔夫特理工大学）；Hospital of the University of Pennsylvania（宾夕法尼亚大学医院）；Ghent University（根特大学）；University of Pennsylvania（宾夕法尼亚大学）
Conditional Latent Diffusion Model with Fourier-based Motion Modelling for Virtual Population Synthesis
Centre for Computational Imaging and Modelling in Medicine (CIMIM)（计算医学成像与建模中心）；University of Manchester（曼彻斯特大学）；Christabel Pankhurst Institute（克里斯塔贝尔·潘克赫斯特研究所）；Department of Computer Science（计算机科学系）；Division of Informatics, Imaging & Data Sciences（信息学、成像与数据科学分会）；Department of Electrical & Electronic Engineering（电子与电气工程系）；NIHR Manchester Biomedical Research Centre, Manchester Academic Health Sciences Centre, University of Manchester（尼日利亚卫生研究委员会曼彻斯特生物医学研究中心、曼彻斯特学术健康科学中心、曼彻斯特大学）
Imaging technology center, Fujifilm corporation（富士胶片公司成像技术中心）；Center for Artificial Intelligence Research, University of Tsukuba（筑波大学人工智能研究中心）；Department of computer science and engineering, Waseda university（早稻田大学计算机科学与工程系）
Many CT slice images are stored with large slice intervals to reduce storage size in clinical practice. This leads to low resolution perpendicular to the slice images (i.e., z-axis), which is insufficient for 3D visualization or image analysis. In this paper, we present a novel architecture based on conditional Generative Adversarial Networks (cGANs) with the goal of generating high resolution images of main body parts including head, chest, abdomen and legs. However, GANs are known to have a difficulty with generating a diversity of patterns due to a phenomena known as mode collapse. To overc…
Montreal Institute for Learning Algorithms（蒙特利尔学习算法研究所）；Imagia Cybernetics（Imagia Cybernetics 公司）
Inspired by the combination of feedforward and iterative computations in the virtual cortex, and taking advantage of the ability of denoising autoencoders to estimate the score of a joint distribution, we propose a novel approach to iterative inference for capturing and exploiting the complex joint distribution of output variables conditioned on some input variables. This approach is applied to image pixel-wise segmentation, with the estimated conditional score used to perform gradient ascent towards a mode of the estimated conditional distribution. This extends previous work on score estimati…
University of Maryland, College Park（马里兰大学帕克分校）；DEVCOM Army Research Laboratory（美国陆军作战能力发展司令部陆军研究实验室）
To effectively interrogate UAV-based images for detecting objects of interest, such as humans, it is essential to acquire large-scale UAV-based datasets that include human instances with various poses captured from widely varying viewing angles. As a viable alternative to laborious and costly data curation, we introduce Progressive Transformation Learning (PTL), which gradually augments a training dataset by adding transformed virtual images with enhanced realism. Generally, a virtual2real transformation generator in the conditional GAN framework suffers from quality degradation when a large d…
Previous virtual try-on methods usually focus on aligning a clothing item with a person, limiting their ability to exploit the complex pose, shape and skin color of the person, as well as the overall structure of the clothing, which is vital to photo-realistic virtual try-on. To address this potential weakness, we propose a fill in fabrics (FIFA) model, a self-supervised conditional generative adversarial network based framework comprised of a Fabricator and a unified virtual try-on pipeline with a Segmenter, Warper and Fuser. The Fabricator aims to reconstruct the clothing image when provided…
HistDiT: A Structure-Aware Latent Conditional Diffusion Model for High-Fidelity Virtual Staining in Histopathology
Edge Hill University（埃奇希尔大学）；University of Nottingham Malaysia（诺丁汉大学马来西亚分校）；University of Southampton Malaysia（南安普顿大学马来西亚分校）；Cancer Research Malaysia（马来西亚癌症研究中心）
We present a generative adversarial learning approach to synthesize gaze behavior of a given personality. We train the model using an existing data set that comprises eye-tracking data and personality traits of 42 participants performing an everyday task. Given the values of Big-Five personality traits (openness, conscientiousness, extroversion, agreeableness, and neuroticism), our model generates time series data consisting of gaze target, blinking times, and pupil dimensions. We use the generated data to synthesize the gaze motion of virtual agents on a game engine.
Leveraging Adversarial Learning for Pathological Fidelity in Virtual Staining
INESC TEC, Faculdade de Engenharia, Universidade do Porto, Porto, Portugal（INESC TEC，工程学院， Porto 大学，Porto，葡萄牙）；The Netherlands Cancer Institute, Amsterdam, The Netherlands（荷兰癌症研究所，阿姆斯特丹，荷兰）；IMP Diagnostics, Porto, Portugal（IMP 诊断公司，Porto，葡萄牙）
Queen Mary University of London（伦敦玛丽女王大学）；Karolinska University Hospital（卡罗林斯卡大学医院）
Histopathological cancer diagnosis is based on visual examination of stained tissue slides. Hematoxylin and eosin (H\&E) is a standard stain routinely employed worldwide. It is easy to acquire and cost effective, but cells and tissue components show low-contrast with varying tones of dark blue and pink, which makes difficult visual assessments, digital image analysis, and quantifications. These limitations can be overcome by IHC staining of target proteins of the tissue slide. IHC provides a selective, high-contrast imaging of cells and tissue components, but their use is largely limited by a …
UTN- Universidad Tecnológica Nacional（阿根廷国家科技大学）；Famaf- Facultad de Matemática Astronomía y Física, Universidad Nacional de Córdoba（科尔多瓦国立大学数学天文物理学院）；Facultad de Ingeniería, FCEFyN, Universidad Nacional de Córdoba（科尔多瓦国立大学工程学院与自然科学与精确科学学院）；Universidad Nacional del Sur（南方国立大学）
In this paper, we study statistical classification accuracy of two different Markov field environments for pixelwise image segmentation, considering the labels of the image as hidden states and solving the estimation of such labels as a solution of the MAP equation. The emission distribution is assumed the same in all models, and the difference lays in the Markovian prior hypothesis made over the labeling random field. The a priori labeling knowledge will be modeled with a) a second order anisotropic Markov Mesh and b) a classical isotropic Potts model. Under such models, we will consider thre…
Simulating Dynamic Tumor Contrast Enhancement in Breast MRI using Conditional Generative Adversarial Networks
Barcelona Artificial Intelligence in Medicine Lab (BCN-AIM)（巴塞罗那人工智能医学实验室）；Departament de Matemàtiques i Informàtica, Universitat de Barcelona（巴塞罗那大学数学与计算机科学系）；Institute of Machine Learning in Biomedical Imaging, Helmholtz Munich（生物医学成像机器学习研究所，海德堡慕尼黑）；School of Computation, Information and Technology, Technical University of Munich（慕尼黑技术大学计算、信息与技术学院）；Department of Oncology-Pathology, Karolinska Institutet（卡罗林斯卡研究所肿瘤-病理学系）；School of Biomedical Engineering & Imaging Sciences, King’s College London（伦敦国王学院生物医学工程与成像科学学院）
Tel Aviv University（特拉维夫大学）；Sheba Medical Center（谢巴医疗中心）
In this work we present a novel system for PET estimation using CT scans. We explore the use of fully convolutional networks (FCN) and conditional generative adversarial networks (GAN) to export PET data from CT data. Our dataset includes 25 pairs of PET and CT scans where 17 were used for training and 8 for testing. The system was tested for detection of malignant tumors in the liver region. Initial results look promising showing high detection performance with a TPR of 92.3% and FPR of 0.25 per case. Future work entails expansion of the current system to the entire body using a much larger d…
