Recently, interest in MR-only treatment planning using synthetic CTs (synCTs) has grown rapidly in radiation therapy. However, developing class solutions for medical images that contain atypical anatomy remains a major limitation. In this paper, we propose a novel spatial attention-guided generative adversarial network (attention-GAN) model to generate accurate synCTs using T1-weighted MRI images as the input to address atypical anatomy. Experimental results on fifteen brain cancer patients show that attention-GAN outperformed existing synCT models and achieved an average MAE of 85.22$\pm$12.08, 232.41$\pm$60.86, 246.38$\pm$42.67 Hounsfield units between synCT and CT-SIM across the entire head, bone and air regions, respectively. Qualitative analysis shows that attention-GAN has the ability to use spatially focused areas to better handle outliers, areas with complex anatomy or post-surgical regions, and thus offer strong potential for supporting near real-time MR-only treatment planning.
Department of Biomedical Engineering, Yale University（耶鲁大学生物医学工程系）；Department of Radiology & Biomedical Imaging, Yale School of Medicine（耶鲁大学医学院放射科与生物医学成像系）；Department of Electrical Engineering, Yale University（耶鲁大学电气工程系）；Department of Statistics & Data Science, Yale University（耶鲁大学统计与数据科学系）；Yale School of Medicine（耶鲁大学医学院）
Domain Adaptation (DA) has the potential to greatly help the generalization of deep learning models. However, the current literature usually assumes to transfer the knowledge from the source domain to a specific known target domain. Domain Agnostic Learning (DAL) proposes a new task of transferring knowledge from the source domain to data from multiple heterogeneous target domains. In this work, we propose the Domain-Agnostic Learning framework with Anatomy-Consistent Embedding (DALACE) that works on both domain-transfer and task-transfer to learn a disentangled representation, aiming to not o…
University of Edinburgh（爱丁堡大学）；Canon Medical Research Europe（佳能医疗欧洲研究中心）；The Alan Turing Institute（艾伦·图灵研究所）
Acquiring annotated data at scale with rare diseases or conditions remains a challenge. It would be extremely useful to have a method that controllably synthesizes images that can correct such underrepresentation. Assuming a proper latent representation, the idea of a "latent vector arithmetic" could offer the means of achieving such synthesis. A proper representation must encode the fidelity of the input data, preserve invariance and equivariance, and permit arithmetic operations. Motivated by the ability to disentangle images into spatial anatomy (tensor) factors and accompanying imaging (ve…
Johns Hopkins University（约翰斯·霍普金斯大学）；Johns Hopkins School of Medicine（约翰斯·霍普金斯大学医学院）；National Institute on Aging, National Institute of Health（美国国立卫生研究院国家老龄化研究所）
Disentangling anatomical and contrast information from medical images has gained attention recently, demonstrating benefits for various image analysis tasks. Current methods learn disentangled representations using either paired multi-modal images with the same underlying anatomy or auxiliary labels (e.g., manual delineations) to provide inductive bias for disentanglement. However, these requirements could significantly increase the time and cost in data collection and limit the applicability of these methods when such data are not available. Moreover, these methods generally do not guarantee …
A large amount of manual segmentation is typically required to train a robust segmentation network so that it can segment objects of interest in a new imaging modality. The manual efforts can be alleviated if the manual segmentation in one imaging modality (e.g., CT) can be utilized to train a segmentation network in another imaging modality (e.g., CBCT/MRI/PET). In this work, we developed an anatomy-constrained contrastive synthetic segmentation network (AccSeg-Net) to train a segmentation network for a target imaging modality without using its ground truth. Specifically, we proposed to use a…
Unified Multi-Modal Image Synthesis for Missing Modality Imputation
School of Biomedical Engineering, Division of Life Sciences and Medicine, University of Science and Technology of China (USTC)（中国科学技术大学生命科学与医学部生物医学工程学院）；Suzhou Institute for Advance Research, USTC（中国科学技术大学苏州高等研究院）
Z 2 Sky Technologies Inc.（Z 2 天工科技公司）；University of Rochester（罗切斯特大学）；Key Lab of Intelligent Information Processing of Chinese Academy of Sciences (CAS), Institute of Computing Technology, CAS（中国科学院计算技术研究所 中国科学院智能信息处理重点实验室）；University of Science and Technology of China（中国科学技术大学）；Suzhou Institute for Advance Research, University of Science and Technology of China（中国科学技术大学苏州高等研究院）
Spinal surgery planning necessitates automatic segmentation of vertebrae in cone-beam computed tomography (CBCT), an intraoperative imaging modality that is widely used in intervention. However, CBCT images are of low-quality and artifact-laden due to noise, poor tissue contrast, and the presence of metallic objects, causing vertebra segmentation, even manually, a demanding task. In contrast, there exists a wealth of artifact-free, high quality CT images with vertebra annotations. This motivates us to build a CBCT vertebra segmentation model using unpaired CT images with annotations. To overco…
seg2med: a bridge from artificial anatomy to multimodal medical images
Computer Assisted Clinical Medicine, Medical Faculty Mannheim, Heidelberg University（计算机辅助临床医学，曼海姆医学院，海德堡大学）；Pattern Recognition Lab, Friedrich-Alexander-University Erlangen-Nuremberg（模式识别实验室，埃尔兰根-纽伦堡弗里德里希-亚历山大大学）；Department of Radiology and Nuclear Medicine, University Medical Center Mannheim（放射学与核医学系，曼海姆大学医学中心）；Mannheim Institute for Intelligent Systems in Medicine, Medical Faculty Mannheim, Heidelberg University（曼海姆智能医学研究所，曼海姆医学院，海德堡大学）；Optical Bioimaging Laboratory, Department of Biomedical Engineering, College of Design and Engineering, National University of Singapore（光学生物成像实验室，生物医学工程系，设计与工程学院，新加坡国立大学）
Yale University（耶鲁大学）；Yale School of Medicine（耶鲁大学医学院）；University of Science and Technology of China（中国科学技术大学）；Suzhou Institute for Advanced Research, University of Science and Technology of China（中国科学技术大学苏州高等研究院）；Chinese Academy of Sciences（中国科学院）
Multimodal image registration has many applications in diagnostic medical imaging and image-guided interventions, such as Transcatheter Arterial Chemoembolization (TACE) of liver cancer guided by intraprocedural CBCT and pre-operative MR. The ability to register peri-procedurally acquired diagnostic images into the intraprocedural environment can potentially improve the intra-procedural tumor targeting, which will significantly improve therapeutic outcomes. However, the intra-procedural CBCT often suffers from suboptimal image quality due to lack of signal calibration for Hounsfield unit, limi…
Inter-scanner and inter-protocol discrepancy in MRI datasets are known to lead to significant quantification variability. Hence image-to-image or scanner-to-scanner translation is a crucial frontier in the area of medical image analysis with a lot of potential applications. Nonetheless, a significant percentage of existing algorithms cannot explicitly exploit and preserve texture details from target scanners and offers individual solutions towards specialized task-specific architectures. In this paper, we design a multi-scale texture transfer to enrich the reconstruction images with more detai…
Johns Hopkins University（约翰斯·霍普金斯大学）；Johns Hopkins University Applied Physics Laboratory（约翰斯·霍普金斯大学应用物理实验室）；Auris Health, Inc.（奥里斯健康公司）；Johns Hopkins Medicine（约翰斯·霍普金斯医学院）；Nara Institute of Science and Technology（奈良科学技术研究所）；University of Texas（德克萨斯大学）；Texas Orthopedics（德克萨斯骨科医院）
Fluoroscopy is the standard imaging modality used to guide hip surgery and is therefore a natural sensor for computer-assisted navigation. In order to efficiently solve the complex registration problems presented during navigation, human-assisted annotations of the intraoperative image are typically required. This manual initialization interferes with the surgical workflow and diminishes any advantages gained from navigation. We propose a method for fully automatic registration using annotations produced by a neural network. Neural networks are trained to simultaneously segment anatomy and ide…
NVIDIA（英伟达）；DeepMind（深度思维）；Oxford Nanopore Technologies（牛津纳米孔技术公司）；University College London（伦敦大学学院）
Human anatomy, morphology, and associated diseases can be studied using medical imaging data. However, access to medical imaging data is restricted by governance and privacy concerns, data ownership, and the cost of acquisition, thus limiting our ability to understand the human body. A possible solution to this issue is the creation of a model able to learn and then generate synthetic images of the human body conditioned on specific characteristics of relevance (e.g., age, sex, and disease status). Deep generative models, in the form of neural networks, have been recently used to create synthe…
Deep Learning-Based Cross-Anatomy CT Synthesis Using Adapted nnResU-Net with Anatomical Feature Prioritized Loss
Univ. Bordeaux, CNRS, LaBRI, UMR 5800, F-33400 Talence, France（波尔多大学，法国国家科学研究中心，LaBRI研究院，UMR 5800）；Universidad Autónoma de Madrid, Madrid, 28049, Spain（马德里自治大学，西班牙）；Pazmany Peter Catholic University, Budapest, 1083, Hungary（佩斯大学，匈牙利）；RIKEN Center for Integrative Medical Sciences, Medical Data Deep Learning Team, Tokyo, Japan（日本再生医学科学中心，医学数据深度学习团队）
