Bidirectional Mapping Generative Adversarial Networks for Brain MR to PET Synthesis
Shenzhen Institutes of Advanced Technology, Chinese Academy of Sciences（中国科学院深圳先进技术研究院）；Shenzhen University（深圳大学）；Central South University（中南大学）；Harbin Engineering University（哈尔滨工程大学）
Fusing multi-modality medical images, such as MR and PET, can provide various anatomical or functional information about human body. But PET data is always unavailable due to different reasons such as cost, radiation, or other limitations. In this paper, we propose a 3D end-to-end synthesis network, called Bidirectional Mapping Generative Adversarial Networks (BMGAN), where image contexts and latent vector are effectively used and jointly optimized for brain MR-to-PET synthesis. Concretely, a bidirectional mapping mechanism is designed to embed the semantic information of PET images into the high dimensional latent space. And the 3D DenseU-Net generator architecture and the extensive objective functions are further utilized to improve the visual quality of synthetic results. The most appealing part is that the proposed method can synthesize the perceptually realistic PET images while preserving the diverse brain structures of different subjects. Experimental results demonstrate that the performance of the proposed method outperforms other competitive cross-modality synthesis methods in terms of quantitative measures, qualitative displays, and classification evaluation.
Zhejiang University（浙江大学）；Binjiang Institute of Zhejiang University（浙江大学滨江研究院）；Hangzhou Universal Medical Imaging Diagnostic Center（杭州全景医学影像诊断中心）
18F-fluorodeoxyglucose (18F-FDG) Positron Emission Tomography (PET) imaging usually needs a full-dose radioactive tracer to obtain satisfactory diagnostic results, which raises concerns about the potential health risks of radiation exposure, especially for pediatric patients. Reconstructing the low-dose PET (L-PET) images to the high-quality full-dose PET (F-PET) ones is an effective way that both reduces the radiation exposure and remains diagnostic accuracy. In this paper, we propose a resource-efficient deep learning framework for L-PET reconstruction and analysis, referred to as transGAN-S…
NVIDIA Corporation（英伟达公司）；Seoul National University（首尔大学）
Synthesizing medical images, such as PET, is a challenging task due to the fact that the intensity range is much wider and denser than those in photographs and digital renderings and are often heavily biased toward zero. Above all, intensity values in PET have absolute significance, and are used to compute parameters that are reproducible across the population. Yet, usually much manual adjustment has to be made in pre-/post- processing when synthesizing PET images, because its intensity ranges can vary a lot, e.g., between -100 to 1000 in floating point values. To overcome these challenges, we…
Corporate Research and Development Center, Toshiba Corporation（东芝公司企业研发中心）
Generative adversarial networks (GANs) have shown promise for various problems including anomaly detection. When anomaly detection is performed using GAN models that learn only the features of normal data samples, data that are not similar to normal data are detected as abnormal samples. The present approach is developed by employing a dual-encoder in a bidirectional GAN architecture that is trained simultaneously with a generator and a discriminator network. Through the learning mechanism, the proposed method aims to reduce the problem of bad cycle consistency, in which a bidirectional GAN mi…
NeuroFlow: Toward Unified Visual Encoding and Decoding from Neural Activity
Shanghai Artificial Intelligence Laboratory（上海人工智能实验室）；University of Hong Kong（香港大学）；Shenzhen Loop Area Institute（深圳河套学院）；Tsinghua University（清华大学）；Chinese University of Hong Kong（香港中文大学）
In recent days, Convolutional Neural Networks (CNN) have demonstrated impressive performance in medical image analysis. However, there is a lack of clear understanding of why and how the Convolutional Neural Network performs so well for image analysis task. How CNN analyzes an image and discriminates among samples of different classes are usually considered as non-transparent. As a result, it becomes difficult to apply CNN based approaches in clinical procedures and automated disease diagnosis systems. In this paper, we consider this issue and work on visualizing and understanding the decision…
Attenuation and scatter correction (AC) is crucial for quantitative Positron Emission Tomography (PET) imaging. Recently, direct application of AC in the image domain using deep learning approaches has been proposed for the hybrid PET/MR and dedicated PET systems that lack accompanying transmission or anatomical imaging. This study set out to investigate deep learning-based AC in the image domain using different input settings.
Functional Imaging Constrained Diffusion for Brain PET Synthesis from Structural MRI
University of North Carolina at Chapel Hill（北卡罗来纳大学教堂山分校）；Joint Department of Biomedical Engineering, University of North Carolina at Chapel Hill and North Carolina State University（北卡罗来纳大学教堂山分校与北卡罗来纳州立大学生物医学工程联合系）；Shanghai Mental Health Center, Shanghai Jiao Tong University School of Medicine（上海交通大学医学院附属精神卫生中心）
Stanford University（斯坦福大学）；GE Healthcare（通用电气医疗集团）；University of California Riverside（加州大学河滨分校）
Accurate quantification of cerebral blood flow (CBF) is essential for the diagnosis and assessment of a wide range of neurological diseases. Positron emission tomography (PET) with radiolabeled water (15O-water) is considered the gold-standard for the measurement of CBF in humans. PET imaging, however, is not widely available because of its prohibitive costs and use of short-lived radiopharmaceutical tracers that typically require onsite cyclotron production. Magnetic resonance imaging (MRI), in contrast, is more readily accessible and does not involve ionizing radiation. This study presents a…
Stanford University（斯坦福大学）；GE Healthcare（通用电气医疗集团）；University of California Riverside（加州大学河滨分校）；National Taiwan University（台湾大学）
Accurate quantification of cerebral blood flow (CBF) is essential for the diagnosis and assessment of cerebrovascular diseases such as Moyamoya, carotid stenosis, aneurysms, and stroke. Positron emission tomography (PET) is currently regarded as the gold standard for the measurement of CBF in the human brain. PET imaging, however, is not widely available because of its prohibitive costs, use of ionizing radiation, and logistical challenges, which require a co-localized cyclotron to deliver the 2 min half-life Oxygen-15 radioisotope. Magnetic resonance imaging (MRI), in contrast, is more readil…
Revolutionizing Disease Diagnosis with simultaneous functional PET/MR and Deeply Integrated Brain Metabolic, Hemodynamic, and Perfusion Networks
School of Biomedical Engineering, ShanghaiTech University（上海科技大学生物医学工程学院）；Zhongshan Hospital（中山医院）；State Key Laboratory of Advanced Medical Materials and Devices, ShanghaiTech University（上海科技大学先进医用材料与器件国家重点实验室）；Shanghai Clinical Research and Trial Center（上海临床研究与试验中心）
University of Electronic Science and Technology of China（电子科技大学）；Shanghai AI Laboratory（上海人工智能实验室）；Sichuan Cancer Hospital and Institute（四川省肿瘤医院研究所）；Sichuan University West China Hospital（四川大学华西医院）；SenseTime Research（商汤科技研究院）
Accurate segmentation of Anatomical brain Barriers to Cancer spread (ABCs) plays an important role for automatic delineation of Clinical Target Volume (CTV) of brain tumors in radiotherapy. Despite that variants of U-Net are state-of-the-art segmentation models, they have limited performance when dealing with ABCs structures with various shapes and sizes, especially thin structures (e.g., the falx cerebri) that span only few slices. To deal with this problem, we propose a High and Multi-Resolution Network (HMRNet) that consists of a multi-scale feature learning branch and a high-resolution bra…
Leveraging healthy population variability in deep learning unsupervised anomaly detection in brain FDG PET
Sorbonne Université（索邦大学）；Institut du Cerveau - Paris Brain Institute - ICM（大脑研究所 - 巴黎大脑研究所 - ICM）；CNRS（法国国家科学研究中心）；Inria（法国国家信息与自动化研究所）；Inserm（法国国家健康与医学研究院）；AP-HP（巴黎公共医疗集团）；Hôpital de la Pitié Salpêtrière（皮蒂-萨尔皮特里耶医院）
针对脑部 FDG PET 无监督异常检测中残差图像方法的局限性，提出受 Z-scores 启发、利用健康人群变异性的新方法，在 ADNI 数据库上证明其能准确识别阿尔茨海默病相关异常。
Lehigh University（里海大学）；University of Pennsylvania（宾夕法尼亚大学）
Identification of brain regions related to the specific neurological disorders are of great importance for biomarker and diagnostic studies. In this paper, we propose an interpretable Graph Convolutional Network (GCN) framework for the identification and classification of Alzheimer's disease (AD) using multi-modality brain imaging data. Specifically, we extended the Gradient Class Activation Mapping (Grad-CAM) technique to quantify the most discriminative features identified by GCN from brain connectivity patterns. We then utilized them to find signature regions of interest (ROIs) by detecting…
Pseudo-MRI-Guided PET Image Reconstruction Method Based on a Diffusion Probabilistic Model
Washington University in St. Louis（圣路易斯华盛顿大学）；Yale University（耶鲁大学）；Western University（西安大略大学）；Lawson Health Research Institute（劳森健康研究所）；McGill University（麦吉尔大学）；Siemens Healthineers AG（西门子医疗保健公司）；Mallinckrodt Institute of Radiology（马林克罗特放射学研究所）
Positron emission tomography (PET) suffers from severe resolution limitations which limit its quantitative accuracy. In this paper, we present a super-resolution (SR) imaging technique for PET based on convolutional neural networks (CNNs). To facilitate the resolution recovery process, we incorporate high-resolution (HR) anatomical information based on magnetic resonance (MR) imaging. We introduce the spatial location information of the input image patches as additional CNN inputs to accommodate the spatially-variant nature of the blur kernels in PET. We compared the performance of shallow (3-…
Technical University Munich（慕尼黑工业大学）；Ludwig Maximilians University Munich（慕尼黑大学）；Johns Hopkins University（约翰斯·霍普金斯大学）
Alzheimer's disease (AD) is the most common cause of dementia. An early detection is crucial for slowing down the disease and mitigating risks related to the progression. While the combination of MRI and FDG-PET is the best image-based tool for diagnosis, FDG-PET is not always available. The reliable detection of Alzheimer's disease with only MRI could be beneficial, especially in regions where FDG-PET might not be affordable for all patients. To this end, we propose a multi-task method based on U-Net that takes T1-weighted MR images as an input to generate synthetic FDG-PET images and classif…
Evaluation of pseudo-healthy image reconstruction for anomaly detection with deep generative models: Application to brain FDG PET
Sorbonne Université（索邦大学）；Institut du Cerveau - Paris Brain Institute - ICM（巴黎大脑研究所）；CNRS（法国国家科学研究中心）；Inria（法国国家信息与自动化研究所）；Inserm（法国国家健康与医学研究院）；AP-HP（巴黎公立医院集团）；Hôpital de la Pitié Salpêtrière（皮提耶-萨勒佩特里医院）
Zhejiang University（浙江大学）；Massachusetts General Hospital（麻省总医院）；Harvard Medical School（哈佛医学院）；First Hospital of Shanxi Medical University（山西医科大学第一医院）
Patlak model is widely used in 18F-FDG dynamic positron emission tomography (PET) imaging, where the estimated parametric images reveal important biochemical and physiology information. Because of better noise modeling and more information extracted from raw sinogram, direct Patlak reconstruction gains its popularity over the indirect approach which utilizes reconstructed dynamic PET images alone. As the prerequisite of direct Patlak methods, raw data from dynamic PET are rarely stored in clinics and difficult to obtain. In addition, the direct reconstruction is time-consuming due to the bottl…
Multi-modal Graph Neural Network for Early Diagnosis of Alzheimer's Disease from sMRI and PET Scans
College of Electronics and Information Engineering, Sichuan University（四川大学电子信息工程学院）；School of Computer Science and Engineering, Nanyang Technological University（南洋理工大学计算机科学与工程学院）
Positron emission tomography(PET) image reconstruction is an ill-posed inverse problem and suffers from high level of noise due to limited counts received. Recently deep neural networks especially convolutional neural networks(CNN) have been successfully applied to PET image reconstruction. However, the local characteristics of the convolution operator potentially limit the image quality obtained by current CNN-based PET image reconstruction methods. In this paper, we propose a residual swin-transformer based regularizer(RSTR) to incorporate regularization into the iterative reconstruction fra…
The Hong Kong University of Science and Technology（香港科技大学）
Registration of pre-operative and post-recurrence brain images is often needed to evaluate the effectiveness of brain gliomas treatment. While recent deep learning-based deformable registration methods have achieved remarkable success with healthy brain images, most of them would be unable to accurately align images with pathologies due to the absent correspondences in the reference image. In this paper, we propose a deep learning-based deformable registration method that jointly estimates regions with absent correspondence and bidirectional deformation fields. A forward-backward consistency c…
Enhancing Alzheimer's Diagnosis: Leveraging Anatomical Landmarks in Graph Convolutional Neural Networks on Tetrahedral Meshes
School of Computing and Augmented Intelligence, Arizona State University, Tempe, AZ, USA（计算与增强智能学院，亚利桑那州立大学）；Amazon AGI, Redmond, WA, USA（亚马逊人工智慧部门）；Banner Alzheimer’s Institute, Phoenix, AZ, USA（贝纳尔德阿尔茨海默病研究所）
ClusMFL: A Cluster-Enhanced Framework for Modality-Incomplete Multimodal Federated Learning in Brain Imaging Analysis
School of Science and Engineering, the Chinese University of Hong Kong, Shenzhen（香港中文大学（深圳）理工学院）；Department of Computer Science and Engineering, Lehigh University（利哈伊大学计算机科学与工程系）；Department of Computer Science, Emory University（埃默里大学计算机科学系）
Images from Positron Emission Tomography (PET) deliver functional data such as perfusion and metabolism. On the other hand, images from Magnetic Resonance Imaging (MRI) provide information describing anatomical structures. Fusing the complementary information from the two modalities is helpful in oncology. In this project, we implemented a complete tool allowing semi-automatic MRI-PET registration for small animal imaging in the preclinical studies. A two-stage hierarchical registration approach is proposed. First, a global affine registration is applied. For robust and fast registration, prin…
A Segmentation Framework for Accurate Diagnosis of Amyloid Positivity without Structural Images
Northwest Agricultural and Forestry Science and Technology University（西北农业和林业科学与技术大学）；Liaoning Province Shiyan High School（辽宁省石yan高级中学）；Northeast Yucai School（东北育才学校）；Northeast Yucai Foreign Language School（东北育才外国语学校）
Cycle-Constrained Adversarial Denoising Convolutional Network for PET Image Denoising: Multi-Dimensional Validation on Large Datasets with Reader Study and Real Low-Dose Data
University of Wisconsin-Madison（威斯康星大学麦迪逊分校）；Shandong University（山东大学）；Inception Institute of Artificial Intelligence（起源人工智能研究院）
Positron emission tomography (PET) imaging is an imaging modality for diagnosing a number of neurological diseases. In contrast to Magnetic Resonance Imaging (MRI), PET is costly and involves injecting a radioactive substance into the patient. Motivated by developments in modality transfer in vision, we study the generation of certain types of PET images from MRI data. We derive new flow-based generative models which we show perform well in this small sample size regime (much smaller than dataset sizes available in standard vision tasks). Our formulation, DUAL-GLOW, is based on two invertible …
Implicit neural representations for end-to-end PET reconstruction
Nantes Université（南特大学）；École Centrale Nantes（南特中央理工学院）；CNRS（法国国家科学研究中心）；Centre Hospitalier Universitaire de Nantes（南特大学医院中心）；INSERM（法国国家健康与医学研究院）；Université d'Angers（昂热大学）；CRCI2NA
Sichuan University（四川大学）；MinFound Medical Systems Co., Ltd（明峰医疗系统股份有限公司）
Positron emission tomography (PET) reconstruction has become an ill-posed inverse problem due to low-count projection data, and a robust algorithm is urgently required to improve imaging quality. Recently, the deep image prior (DIP) has drawn much attention and has been successfully applied in several image restoration tasks, such as denoising and inpainting, since it does not need any labels (reference image). However, overfitting is a vital defect of this framework. Hence, many methods have been proposed to mitigate this problem, and DeepRED is a typical representation that combines DIP and …
Shanghai University（上海大学）；Suzhou Institute of Biomedical Engineering and Technology, Chinese Academy of Sciences（中国科学院苏州生物医学工程技术研究所）；University of Science and Technology of China（中国科学技术大学）；Nanjing Medical University（南京医科大学）；The University of Electronic Science and Technology of China（电子科技大学）
FDG-PET reveals altered brain metabolism in individuals with mild cognitive impairment (MCI) and Alzheimer's disease (AD). Some biomarkers derived from FDG-PET by computer-aided-diagnosis (CAD) technologies have been proved that they can accurately diagnosis normal control (NC), MCI, and AD. However, the studies of identification of early MCI (EMCI) and late MCI (LMCI) with FDG-PET images are still insufficient. Compared with studies based on fMRI and DTI images, the researches of the inter-region representation features in FDG-PET images are insufficient. Moreover, considering the variability…
BG-GAN: Generative AI Enable Representing Brain Structure-Function Connections for Alzheimer's Disease
Shenzhen Institutes of Advanced Technology, Chinese Academy of Sciences（中国科学院深圳先进技术研究院）；University of Chinese Academy of Sciences（中国科学院大学）；Stevens Institute of Technology（史蒂文斯理工学院）；Hong Kong Metropolitan University（香港都会大学）；VinUniversity（越南 VinUniversity 大学）；King Fahd University of Petroleum and Minerals（法赫德国王石油矿产大学）；Uppsala University（乌普萨拉大学）；Tiangong University（天津工业大学）
