For a unified analysis of medical images from different modalities, data harmonization using image-to-image (I2I) translation is desired. We study this problem employing an optical coherence tomography (OCT) data set of Spectralis-OCT and Home-OCT images. I2I translation is challenging because the images are unpaired, and a bijective mapping does not exist due to the information discrepancy between both domains. This problem has been addressed by the Contrastive Learning for Unpaired I2I Translation (CUT) approach, but it reduces semantic consistency. To restore the semantic consistency, we support the style decoder using an additional segmentation decoder. Our approach increases the similarity between the style-translated images and the target distribution. Importantly, we improve the segmentation of biomarkers in Home-OCT images in an unsupervised domain adaptation scenario. Our data harmonization approach provides potential for the monitoring of diseases, e.g., age related macular disease, using different OCT devices.
South China University of Technology（华南理工大学）；University of Utah（犹他大学）
In medical applications, the same anatomical structures may be observed in multiple modalities despite the different image characteristics. Currently, most deep models for multimodal segmentation rely on paired registered images. However, multimodal paired registered images are difficult to obtain in many cases. Therefore, developing a model that can segment the target objects from different modalities with unpaired images is significant for many clinical applications. In this work, we propose a novel two-stream translation and segmentation unified attentional generative adversarial network (U…
Boston University（波士顿大学）；Google Research（谷歌研究院）；Georgia Institute of Technology（佐治亚理工学院）
Recent conditional image generation methods produce images of remarkable diversity, fidelity and realism. However, the majority of these methods allow conditioning only on labels or text prompts, which limits their level of control over the generation result. In this paper, we introduce MaskSketch, an image generation method that allows spatial conditioning of the generation result using a guiding sketch as an extra conditioning signal during sampling. MaskSketch utilizes a pre-trained masked generative transformer, requiring no model training or paired supervision, and works with input sketch…
Imperial College London（帝国理工学院）；University of Chinese Academy of Sciences（中国科学院大学）；Institute of Computing Technology, CAS（中国科学院计算技术研究所）；Peng Cheng Laboratory（鹏城实验室）；Southest University（东南大学）；Nankai University（南开大学）
Although chest X-ray (CXR) offers a 2D projection with overlapped anatomies, it is widely used for clinical diagnosis. There is clinical evidence supporting that decomposing an X-ray image into different components (e.g., bone, lung and soft tissue) improves diagnostic value. We hereby propose a decomposition generative adversarial network (DecGAN) to anatomically decompose a CXR image but with unpaired data. We leverage the anatomy knowledge embedded in CT, which features a 3D volume with clearly visible anatomies. Our key idea is to embed CT priori decomposition knowledge into the latent spa…
Anatomy-preserving unpaired cone-beam CT refinement for image-guided radiotherapy using pseudo-label guided diffusion
Shenzhen Institutes of Advanced Technology, Chinese Academy of Sciences（中国科学院深圳先进技术研究院）；Southern University of Science and Technology（南方科技大学）；Huashan Hospital, Fudan University（复旦大学附属华山医院）
LAND: Lung and Nodule Diffusion for 3D Chest CT Synthesis with Anatomical Guidance
Eurecat, Centre Tecnològic de Catalunya（埃鲁卡特技术中心，加泰罗尼亚技术中心）；Dept. de Matemàtiques i Informàtica, Universitat de Barcelona（巴塞罗那大学数学与计算机系）；Barcelona Supercomputing Center (BSC)（巴塞罗那超级计算中心）
Cross-contrast image translation is an important task for completing missing contrasts in clinical diagnosis. However, most existing methods learn separate translator for each pair of contrasts, which is inefficient due to many possible contrast pairs in real scenarios. In this work, we propose a unified Hyper-GAN model for effectively and efficiently translating between different contrast pairs. Hyper-GAN consists of a pair of hyper-encoder and hyper-decoder to first map from the source contrast to a common feature space, and then further map to the target contrast image. To facilitate the tr…
Conditional image generation is effective for diverse tasks including training data synthesis for learning-based computer vision. However, despite the recent advances in generative adversarial networks (GANs), it is still a challenging task to generate images with detailed conditioning on object shapes. Existing methods for conditional image generation use category labels and/or keypoints and are only give limited control over object categories. In this work, we present SCGAN, an architecture to generate images with a desired shape specified by an input normal map. The shape-conditioned image …
NCT/UCC Dresden（德累斯顿国家肿瘤疾病中心/大学癌症中心）；CeTI, TU Dresden（德累斯顿工业大学触觉互联网卓越中心）；University Hospital Dresden（德累斯顿大学医院）
Research in unpaired video translation has mainly focused on short-term temporal consistency by conditioning on neighboring frames. However for transfer from simulated to photorealistic sequences, available information on the underlying geometry offers potential for achieving global consistency across views. We propose a novel approach which combines unpaired image translation with neural rendering to transfer simulated to photorealistic surgical abdominal scenes. By introducing global learnable textures and a lighting-invariant view-consistency loss, our method produces consistent translation…
Concordia Institute for Information Systems Engineering（康考迪亚大学信息系统工程研究所）；Concordia University（康考迪亚大学）
Motivated by the lack of publicly available datasets of chest radiographs of positive patients with Coronavirus disease 2019 (COVID-19), we build the first-of-its-kind open dataset of synthetic COVID-19 chest X-ray images of high fidelity using an unsupervised domain adaptation approach by leveraging class conditioning and adversarial training. Our contributions are twofold. First, we show considerable performance improvements on COVID-19 detection using various deep learning architectures when employing synthetic images as additional training set. Second, we show how our image synthesis metho…
ProSAC-CT: Progressive Spectral-Anatomical Co-Guided Multi-Stage Diffusion Model for Low-Dose CT Denoising
College of Medicine and Biological Information Engineering, Northeastern University（东北大学医学与生物信息工程学院）；Tohoku University School of Medicine（东北大学医学院）；State Key Laboratory of Oncology in South China, Sun Yat-sen University Cancer Center（华南肿瘤学国家重点实验室，中山大学肿瘤防治中心）；AI Lab, Tohoku University Hospital（东北大学医院人工智能实验室）
Technical University of Munich（慕尼黑工业大学）；Klinikum rechts der Isar（右伊萨尔医院）；University of Zurich（苏黎世大学）；Institut für KI und Informatik in der Medizin（医学人工智能与信息学研究所）；Imperial College London（伦敦帝国学院）
Retrieval-Augmented Anatomical Guidance for Text-to-CT Generation
Unit of Artificial Intelligence and Computer Systems, Department of Engineering, Università Campus Bio-Medico di Roma, Rome, Italy（人工智能与计算机系统单位，工程系，罗马生物医学大学，意大利罗马）；Department of Diagnostics and Intervention, Biomedical Engineering and Radiation Physics, Umeå University, Umeå, Sweden（诊断与介入系，生物医学工程与放射物理，乌梅大学，瑞典乌梅）
MedSyn: Text-guided Anatomy-aware Synthesis of High-Fidelity 3D CT Images
Boston University（波士顿大学）；Stanford University（斯坦福大学）；Carnegie Mellon University（卡内基梅隆大学）；University of Pittsburgh Medical Center（匹兹堡大学医学中心）；University of Pittsburgh（匹兹堡大学）
Structure-Preserving Synthesis: MaskGAN for Unpaired MR-CT Translation
Australian Institute for Machine Learning, University of Adelaide（阿德莱德大学澳大利亚机器学习研究所）；Flinders Health and Medical Research Institute, Flinders University（弗林德斯大学弗林德斯健康与医学研究所）
ContourDiff: Unpaired Medical Image Translation with Structural Consistency
Department of Electrical and Computer Engineering, Duke University（杜克大学电气与计算机工程系）；Department of Biostatistics & Bioinformatics, Duke University（杜克大学生物统计学与生物信息学系）；Department of Radiology, Duke University（杜克大学放射学系）；Department of Computer Science, Duke University（杜克大学计算机科学系）
New York University Tandon School of Engineering（纽约大学坦登工程学院）
Deep networks are now ubiquitous in large-scale multi-center imaging studies. However, the direct aggregation of images across sites is contraindicated for downstream statistical and deep learning-based image analysis due to inconsistent contrast, resolution, and noise. To this end, in the absence of paired data, variations of Cycle-consistent Generative Adversarial Networks have been used to harmonize image sets between a source and target domain. Importantly, these methods are prone to instability, contrast inversion, intractable manipulation of pathology, and steganographic mappings which l…
CT Synthesis with Conditional Diffusion Models for Abdominal Lymph Node Segmentation
Shanghai Jiao Tong University, Shanghai, China（上海交通大学）；Department of Surgical Oncology and General Surgery, Key Laboratory of Precision Diagnosis and Treatment of Gastrointestinal Tumors, Ministry of Education, The First Hospital of China Medical University, Shenyang, China（外科肿瘤科和普通外科，精准诊断与治疗胃肠肿瘤国家重点实验室，教育部，中国医科大学第一附属医院，沈阳，中国）；Department of Radiology, The First Hospital of China Medical University, Shenyang, China（放射科，中国医科大学第一附属医院，沈阳，中国）；Shanghai AI Laboratory, Shanghai, China（上海人工智能实验室，上海，中国）
Guided Neural Schrödinger bridge for Brain MR image synthesis with Limited Data
Hanyang University（翰阳大学）；Hanyang University College of Medicine（翰阳大学医学院）；Chonnam National University Medical School（全南国立大学医学院）；Keimyung University School of Medicine（庆尚大学医学院）；Yonsei Wonju College of Medicine（延世Wonju医学院）；Yonsei University College of Medicine（延世大学医学院）
Retrospective artifact correction (RAC) improves image quality post acquisition and enhances image usability. Recent machine learning driven techniques for RAC are predominantly based on supervised learning and therefore practical utility can be limited as data with paired artifact-free and artifact-corrupted images are typically insufficient or even non-existent. Here we show that unwanted image artifacts can be disentangled and removed from an image via an RAC neural network learned with unpaired data. This implies that our method does not require matching artifact-corrupted data to be eithe…
Freehand Ultrasound Image Simulation with Spatially-Conditioned Generative Adversarial Networks
Centre for Medical Image Computing, University College London（伦敦大学学院医学图像计算中心）；Institute of Biomedical Engineering, University of Oxford（牛津大学生物医学工程研究所）；Department of Diagnostic Imaging, Royal Free London NHS Foundation Trust（伦敦皇家自由NHS基金会信托诊断影像科）
Wavelet-Fusion Diffusion Model for Multimodal Brain MRI Synthesis with Modality and Metadata Conditioning
Department of Psychology & Neuroscience, Auckland University of Technology（心理学与神经科学系，奥克兰技术大学）；Department of Psychiatry, University of Melbourne（精神病学系，墨尔本大学）
Interactive simulation of ultrasound imaging greatly facilitates sonography training. Although ray-tracing based methods have shown promising results, obtaining realistic images requires substantial modeling effort and manual parameter tuning. In addition, current techniques still result in a significant appearance gap between simulated images and real clinical scans. Herein we introduce a novel content-preserving image translation framework (ConPres) to bridge this appearance gap, while maintaining the simulated anatomical layout. We achieve this goal by leveraging both simulated images with …
Conditioning Deep Anatomical Prior Knowledge for Reconstruction of Multispectral Optoacoustic Tomography Images
Technical University of Munich（慕尼黑工业大学）；Central Institute for Translational Cancer Research (TranslaTUM)（转化癌症研究中心 (TranslaTUM)）；School of Computation, Information and Technology, Technical University of Munich（慕尼黑工业大学计算、信息与技术学院）；Institute of Computational Biology, Helmholtz Zentrum München（赫尔姆霍兹中心慕尼黑计算生物学研究所）；Institute of AI for Health, Helmholtz Zentrum München（赫尔姆霍兹中心慕尼黑健康人工智能研究所）；Ludwig-Maximilians-Universität München（慕尼黑大学）
Fetal brain magnetic resonance imaging (MRI) offers exquisite images of the developing brain but is not suitable for anomaly screening. For this ultrasound (US) is employed. While expert sonographers are adept at reading US images, MR images are much easier for non-experts to interpret. Hence in this paper we seek to produce images with MRI-like appearance directly from clinical US images. Our own clinical motivation is to seek a way to communicate US findings to patients or clinical professionals unfamiliar with US, but in medical image analysis such a capability is potentially useful, for in…
Synthetic Cardiac MRI Image Generation using Deep Generative Models
Department of Computer Engineering（计算机工程系）；University of Peradeniya（珀德尼亚大学）；University of Maryland（马里兰大学）；Department of Holistic Systems（整体系统系）；SimulaMet；Oslo, Norway（挪威奥斯陆）
Product-of-Gaussian-Mixture Diffusion Models for Joint Nonlinear MRI Reconstruction
Graz University of Technology（格拉茨技术大学）；École Polytechnique Fédérale de Lausanne（洛桑联邦理工学院）；Biomedical Imaging Group and Center for Biomedical Imaging（生物医学成像组和生物医学成像中心）
Philips Research Paris（飞利浦巴黎研究院）；Institut Imagine, Université Paris Cité（巴黎西岱大学想象研究所）；Université Paris Cité（巴黎西岱大学）；Hôpital Necker Enfants-Malades, APHP（巴黎公共援助医院集团内克儿童病院）；Sorbonne Université（索邦大学）；CNRS, LIP6（法国国家科学研究中心LIP6实验室）；Technology Innovation Institute（技术创新研究所）
Anatomical structures such as blood vessels in contrast-enhanced CT (ceCT) images can be challenging to segment due to the variability in contrast medium diffusion. The combined use of ceCT and contrast-free (CT) CT images can improve the segmentation performances, but at the cost of a double radiation exposure. To limit the radiation dose, generative models could be used to synthesize one modality, instead of acquiring it. The CycleGAN approach has recently attracted particular attention because it alleviates the need for paired data that are difficult to obtain. Despite the great performance…
CSG: A Context-Semantic Guided Diffusion Approach in De Novo Musculoskeletal Ultrasound Image Generation
GE Healthcare（GE医疗）；Science & Technology Organization, GE Healthcare（GE医疗科技组织）；Point of Care Ultrasound & Handheld（床旁超声与 handheld（注：此处保留原文表述，结合上下文为GE医疗相关临床应用部门，按要求提取到机构级别））
Z 2 Sky Technologies Inc.（Z 2 天工科技公司）；University of Rochester（罗切斯特大学）；Key Lab of Intelligent Information Processing of Chinese Academy of Sciences (CAS), Institute of Computing Technology, CAS（中国科学院计算技术研究所 中国科学院智能信息处理重点实验室）；University of Science and Technology of China（中国科学技术大学）；Suzhou Institute for Advance Research, University of Science and Technology of China（中国科学技术大学苏州高等研究院）
Spinal surgery planning necessitates automatic segmentation of vertebrae in cone-beam computed tomography (CBCT), an intraoperative imaging modality that is widely used in intervention. However, CBCT images are of low-quality and artifact-laden due to noise, poor tissue contrast, and the presence of metallic objects, causing vertebra segmentation, even manually, a demanding task. In contrast, there exists a wealth of artifact-free, high quality CT images with vertebra annotations. This motivates us to build a CBCT vertebra segmentation model using unpaired CT images with annotations. To overco…
Evaluation of 3D Counterfactual Brain MRI Generation
Wu Tsai Neurosciences Institute, Stanford University, Stanford, CA, USA Department of Radiology, Stanford University, Stanford, CA, USA Dept. of Psychiatry \& Behavioral Sciences, Stanford University, Stanford, CA, USA Department of Computer Science, Stanford University, Stanford, CA, USA Department of Bioengineering, Stanford University, Stanford, CA, USA Department of Electrical Engineering, Stanford University, Stanford, CA, USA
University of Florida（佛罗里达大学）；Medical Imaging Technologies, Siemens Healthcare（西门子医疗医学影像技术部门）
Synthesized medical images have several important applications, e.g., as an intermedium in cross-modality image registration and as supplementary training samples to boost the generalization capability of a classifier. Especially, synthesized computed tomography (CT) data can provide X-ray attenuation map for radiation therapy planning. In this work, we propose a generic cross-modality synthesis approach with the following targets: 1) synthesizing realistic looking 3D images using unpaired training data, 2) ensuring consistent anatomical structures, which could be changed by geometric distorti…
We compare two conditioning mechanisms based on concatenation and feature-wise modulation to integrate non-imaging information into convolutional neural networks for segmentation of anatomical structures. As a proof-of-concept we provide the distribution of class labels obtained from ground truth masks to ensure strong correlation between the conditioning data and the segmentation maps. We evaluate the methods on the ACDC dataset, and show that conditioning with non-imaging data improves performance of the segmentation networks. We observed conditioning the U-Net architectures was challenging,…
