Magnetic Resonance Imaging (MRI) plays a crucial role in brain disease diagnosis, but it is not always feasible for certain patients due to physical or clinical constraints. Recent studies attempt to synthesize MRI from Computed Tomography (CT) scans; however, low-dose protocols often result in highly sparse CT volumes with poor through-plane resolution, making accurate reconstruction of the full brain MRI volume particularly challenging. To address this, we propose ReBrain, a retrieval-augmented diffusion framework for brain MRI reconstruction. Given any 3D CT scan with limited slices, we first employ a Brownian Bridge Diffusion Model (BBDM) to synthesize MRI slices along the 2D dimension. Simultaneously, we retrieve structurally and pathologically similar CT slices from a comprehensive prior database via a fine-tuned retrieval model. These retrieved slices are used as references, incorporated through a ControlNet branch to guide the generation of intermediate MRI slices and ensure structural continuity. We further account for rare retrieval failures when the database lacks suitable references and apply spherical linear interpolation to provide supplementary guidance. Extensive experiments on SynthRAD2023 and BraTS demonstrate that ReBrain achieves state-of-the-art performance in cross-modal reconstruction under sparse conditions.
LPIXEL Inc.（LPIXEL公司）；University of Cambridge（剑桥大学）；National Institute of Informatics（国立信息学研究所）；National Center for Global Health and Medicine（国立国际医疗研究中心）；Juntendo University（顺天堂大学）；ELTE Eötvös Loránd University（厄特沃什·罗兰大学）；The University of Tokyo（东京大学）
Unsupervised learning can discover various unseen abnormalities, relying on large-scale unannotated medical images of healthy subjects. Towards this, unsupervised methods reconstruct a 2D/3D single medical image to detect outliers either in the learned feature space or from high reconstruction loss. However, without considering continuity between multiple adjacent slices, they cannot directly discriminate diseases composed of the accumulation of subtle anatomical anomalies, such as Alzheimer's Disease (AD). Moreover, no study has shown how unsupervised anomaly detection is associated with eith…
Comparative Analysis of GAN and Diffusion for MRI-to-CT translation
Department of Computer Science, University of Copenhagen, Copenhagen, Denmark（哥本哈根大学计算机科学系）；Department of Oncology, Rigshospitalet, Copenhagen, Denmark（哥本哈根医院肿瘤科）
Research Center for Medical Big Data, National Institute of Informatics（日本情报学研究所医疗大数据研究中心）；LPixel Inc.（LPixel公司）；Graduate School of Information Science and Technology, The University of Tokyo（东京大学信息科学与技术研究生院）；Department of Radiology, University of Cambridge（剑桥大学放射科）；Cancer Research UK Cambridge Centre（英国癌症研究中心剑桥中心）；Department of Artificial Intelligence, ELTE Eötvös Loránd University（厄特沃什罗兰大学人工智能系）；Department of Rehabilitation Medicine, Juntendo University School of Medicine（顺天堂大学医学部康复医学科）；International Research Center for Neurointelligence (WPI-IRCN), The University of Tokyo（东京大学国际神经智能研究中心（WPI-IRCN））
Unsupervised learning can discover various unseen diseases, relying on large-scale unannotated medical images of healthy subjects. Towards this, unsupervised methods reconstruct a single medical image to detect outliers either in the learned feature space or from high reconstruction loss. However, without considering continuity between multiple adjacent slices, they cannot directly discriminate diseases composed of the accumulation of subtle anatomical anomalies, such as Alzheimer's Disease (AD). Moreover, no study has shown how unsupervised anomaly detection is associated with disease stages.…
Amrita School of Engineering（阿姆里塔工程学院）；Amrita Vishwa Vidyapeetham（阿姆里塔大学）；Inception Institute of Artificial Intelligence（起源人工智能研究院）；University of Canberra（堪培拉大学）
Diagnosis of Autism Spectrum Disorder (ASD) using clinical evaluation (cognitive tests) is challenging due to wide variations amongst individuals. Since no effective treatment exists, prompt and reliable ASD diagnosis can enable the effective preparation of treatment regimens. This paper proposes structural Magnetic Resonance Imaging (sMRI)-based ASD diagnosis via an outlier detection approach. To learn Spatio-temporal patterns in structural brain connectivity, a Generative Adversarial Network (GAN) is trained exclusively with sMRI scans of healthy subjects. Given a stack of three adjacent sli…
Image Sciences Institute, University Medical Center Utrecht（乌得勒支大学医学中心图像科学研究所）；Wilhelmina Children’s Hospital, University Medical Center Utrecht（乌得勒支大学医学中心威廉明娜儿童医院）；Brain Center Rudolf Magnus, University Medical Center Utrecht（乌得勒支大学医学中心鲁道夫·马格努斯脑中心）
Automatic neonatal brain tissue segmentation in preterm born infants is a prerequisite for evaluation of brain development. However, automatic segmentation is often hampered by motion artifacts caused by infant head movements during image acquisition. Methods have been developed to remove or minimize these artifacts during image reconstruction using frequency domain data. However, frequency domain data might not always be available. Hence, in this study we propose a method for removing motion artifacts from the already reconstructed MR scans. The method employs a generative adversarial network…
Hierarchical Diffusion Framework for Pseudo-Healthy Brain MRI Inpainting with Enhanced 3D Consistency
The Grainger College of Engineering（格拉inger工程学院）；National Center for Supercomputing Applications（国家超级计算应用中心）；University of Illinois Urbana‑Champaign（伊利诺伊大学厄巴纳-香槟分校）
Sun Yat-Sen University（中山大学）；University of Edinburgh（爱丁堡大学）；Cardiovascular Research Centre, Royal Brompton Hospital（皇家布朗普顿医院心血管研究中心）；National Heart & Lung Institute, Imperial College London（帝国理工学院国家心肺研究所）
The performance of traditional compressive sensing-based MRI (CS-MRI) reconstruction is affected by its slow iterative procedure and noise-induced artefacts. Although many deep learning-based CS-MRI methods have been proposed to mitigate the problems of traditional methods, they have not been able to achieve more robust results at higher acceleration factors. Most of the deep learning-based CS-MRI methods still can not fully mine the information from the k-space, which leads to unsatisfactory results in the MRI reconstruction. In this study, we propose a new deep learning-based CS-MRI reconstr…
Self-Supervised Slice-to-Volume Reconstruction with Gaussian Representations for Fetal MRI
Department of Electrical and Electronic Engineering and I-X, Imperial College London（伦敦帝国理工学院电气与电子工程系及I-X）；Trinity College Institute of Neuroscience, Trinity College Dublin（都柏林圣三一学院神经科学研究所）
University of California, Los Angeles (UCLA)（加利福尼亚大学洛杉矶分校）；The First Affiliated Hospital of China Medical University（中国医科大学附属第一医院）；VoxelCloud, Inc.（体素云公司）
Prostate cancer is the second leading cause of cancer death among men in the United States. The diagnosis of prostate MRI often relies on the accurate prostate zonal segmentation. However, state-of-the-art automatic segmentation methods often fail to produce well-contained volumetric segmentation of the prostate zones since certain slices of prostate MRI, such as base and apex slices, are harder to segment than other slices. This difficulty can be overcome by accounting for the cross-slice relationship of adjacent slices, but current methods do not fully learn and exploit such relationships. I…
MIT（麻省理工学院）；Boston Children’s Hospital（波士顿儿童医院）；Harvard Medical School（哈佛医学院）；University College London（伦敦大学学院）；Massachusetts General Hospital（麻省总医院）
Volumetric reconstruction of fetal brains from multiple stacks of MR slices, acquired in the presence of almost unpredictable and often severe subject motion, is a challenging task that is highly sensitive to the initialization of slice-to-volume transformations. We propose a novel slice-to-volume registration method using Transformers trained on synthetically transformed data, which model multiple stacks of MR slices as a sequence. With the attention mechanism, our model automatically detects the relevance between slices and predicts the transformation of one slice using information from othe…
Introducing DEFORMISE: A deep learning framework for dementia diagnosis in the elderly using optimized MRI slice selection
International Hellenic University（国际希腊大学）；MetaMind Innovations；Aristotle University of Thessaloniki（塞萨洛尼基亚里士多德大学）；Kingston University London（伦敦金斯顿大学）；University of Western Macedonia（西马其顿大学）
Dynamic Magnetic Resonance Imaging (dMRI) is widely used to assess various cardiac conditions such as cardiac motion and blood flow. To accelerate MR acquisition, techniques such as undersampling and Simultaneous Multi-Slice (SMS) are often used. Special reconstruction algorithms are needed to reconstruct multiple SMS image slices from the entangled information. Deep learning (DL)-based methods have shown promising results for single-slice MR reconstruction, but the addition of SMS acceleration raises unique challenges due to the composite k-space signals and the resulting images with strong i…
Meta-learning Slice-to-Volume Reconstruction in Fetal Brain MRI using Implicit Neural Representations
School of Computation, Information and Technology, and School of Medicine and Health, Technical University Munich（计算、信息与技术学院，以及医学与健康学院，技术大学慕尼黑）；CIBM Center for Biomedical Imaging and Radiology Department of Lausanne University Hospital and University of Lausanne（日内瓦大学医院与日内瓦大学生物医学影像与放射学中心部门）；Centre for the Developing Brain, School of Biomedical Engineering and Imaging Sciences, King’s College London（发展脑中心，生物医学工程与影像科学学院，伦敦国王学院）；Biomedical Image Technologies, ETSI Telecomunicación, Universidad Politécnica de Madrid（生物医学影像技术，电信工程学院，马德里理工大学）；Department of Computing, Imperial College London（计算学院，伦敦帝国学院）
Back to Physics: Operator-Guided Generative Paths for SMS MRI Reconstruction
School of Advanced Manufacturing, Nanchang University（先进制造学院，南昌大学）；College of Computer Science and Software Engineering, Shenzhen University（计算机科学与软件工程学院，深圳大学）
Zhejiang University（浙江大学）；Shandong Medical Imaging Research Institute（山东医学影像学研究所）；Cheeloo College of Medicine, Shandong University（山东大学齐鲁医学院）；Shanghai Jiao Tong University（上海交通大学）
Multi-slice magnetic resonance images of the fetal brain are usually contaminated by severe and arbitrary fetal and maternal motion. Hence, stable and robust motion correction is necessary to reconstruct high-resolution 3D fetal brain volume for clinical diagnosis and quantitative analysis. However, the conventional registration-based correction has a limited capture range and is insufficient for detecting relatively large motions. Here, we present a novel Affinity Fusion-based Framework for Iteratively Random Motion (AFFIRM) correction of the multi-slice fetal brain MRI. It learns the sequent…
YOLO-PVC: 2D-to-3D Consolidation of Slice-wise Detections for Volumetric Liver Tumor Localization in MRI
ESME Research Lab（ESME研究实验室）；Université Paris-Est Créteil（巴黎东部克雷泰伊大学）；LRE EPITA（EPITA LRE实验室）；Institute of Space Technology（空间技术研究所）；Henri Mondor University Hospital（亨利·蒙多大学医院）
Image-based brain cancer prediction models, based on radiomics, quantify the radiologic phenotype from magnetic resonance imaging (MRI). However, these features are difficult to reproduce because of variability in acquisition and preprocessing pipelines. Despite evidence of intra-tumor phenotypic heterogeneity, the spatial diversity between different slices within an MRI scan has been relatively unexplored using such methods. In this work, we propose a deep anchor attention aggregation strategy with a Vision Transformer to predict survival risk for brain cancer patients. A Deep Anchor Attentio…
University of Iowa（爱荷华大学）；Philips HealthCare（飞利浦医疗）；University of Wisconsin（威斯康星大学）；GE HealthCare（通用电气医疗）
Free-breathing cardiac MRI schemes are emerging as competitive alternatives to breath-held cine MRI protocols, enabling applicability to pediatric and other population groups that cannot hold their breath. Because the data from the slices are acquired sequentially, the cardiac/respiratory motion patterns may be different for each slice; current free-breathing approaches perform independent recovery of each slice. In addition to not being able to exploit the inter-slice redundancies, manual intervention or sophisticated post-processing methods are needed to align the images post-recovery for qu…
Probabilistic modelling has been an essential tool in medical image analysis, especially for analyzing brain Magnetic Resonance Images (MRI). Recent deep learning techniques for estimating high-dimensional distributions, in particular Variational Autoencoders (VAEs), opened up new avenues for probabilistic modeling. Modelling of volumetric data has remained a challenge, however, because constraints on available computation and training data make it difficult effectively leverage VAEs, which are well-developed for 2D images. We propose a method to model 3D MR brain volumes distribution by combi…
LoGo-MR: Screening Breast MRI for Cancer Risk Prediction by Efficient Omni-Slice Modeling
Netherlands Cancer Institute（荷兰癌症研究所）；Radboud University Medical Center（拉德堡德大学医学中心）；University Medical Center Utrecht（乌得勒支大学医学中心）；Macao Polytechnic University（澳门理工大学）
Indian Institute of Technology Indore（印度理工学院印多尔分校）
Alzheimer's Disease (AD) is a neurodegenerative disease that is one of the significant causes of death in the elderly population. Many deep learning techniques have been proposed to diagnose AD using Magnetic Resonance Imaging (MRI) scans. Predicting AD using 2D slices extracted from 3D MRI scans is challenging as the inter-slice information gets lost. To this end, we propose a novel and lightweight framework termed 'Biceph-Net' for AD diagnosis using 2D MRI scans that model both the intra-slice and inter-slice information. Biceph-Net has been experimentally shown to perform similar to other S…
Faculty of Computer Science, Otto-von-Guericke University Magdeburg（马格德堡奥托·冯·格里克大学计算机科学学院）；Clinic for Radiology and Nuclear Medicine, University Hospital Magdeburg（马格德堡大学医院放射与核医学诊所）
Purpose: We aim to develop a robust 4D MRI method for large FOVs enabling the extraction of irregular respiratory motion that is readily usable with all MRI machines and thus applicable to support a wide range of interventional settings. Method: We propose a 4D MRI reconstruction method to capture an arbitrary number of breathing states. It uses template updates in navigator slices and search regions for fast and robust vessel cross-section tracking. It captures FOVs of 255 mm x 320 mm x 228 mm at a spatial resolution of 1.82 mm x 1.82 mm x 4mm and temporal resolution of 200ms. A total of 37 4…
University of Electronic Science and Technology of China（电子科技大学）；King’s College London（伦敦国王学院）；University Hospitals KU Leuven（鲁汶大学附属医院）；University College London（伦敦大学学院）；SenseTime Research（商汤科技研究院）
Segmentation of the fetal brain from stacks of motion-corrupted fetal MRI slices is important for motion correction and high-resolution volume reconstruction. Although Convolutional Neural Networks (CNNs) have been widely used for automatic segmentation of the fetal brain, their results may still benefit from interactive refinement for challenging slices. To improve the efficiency of interactive refinement process, we propose an Uncertainty-Guided Interactive Refinement (UGIR) framework. We first propose a grouped convolution-based CNN to obtain multiple automatic segmentation predictions with…
In cardiac magnetic resonance imaging, fully-automatic segmentation of the heart enables precise structural and functional measurements to be taken, e.g. from short-axis MR images of the left-ventricle. In this work we propose a recurrent fully-convolutional network (RFCN) that learns image representations from the full stack of 2D slices and has the ability to leverage inter-slice spatial dependences through internal memory units. RFCN combines anatomical detection and segmentation into a single architecture that is trained end-to-end thus significantly reducing computational time, simplifyin…
SAGCNet: Spatial-Aware Graph Completion Network for Missing Slice Imputation in Population CMR Imaging
School of Engineering, College of Engineering and Physical Sciences（工程学院，工程与物理科学学院）；University of Birmingham（伯明翰大学）；Oxford Center for Clinical Magnetic Resonance Research (OCMR)（牛津临床磁共振研究中心）；University of Oxford（牛津大学）；John Radcliffe Hospital（约翰·拉德克利夫医院）；Johns Hopkins University School of Medicine（约翰霍普金斯大学医学院）；Queen Mary University London（伦敦女王大学）；Barts Health NHS Trust（巴特勒医院 NHS 信托）
Amsterdam University Medical Centers（阿姆斯特丹大学医学中心）；University of Amsterdam（阿姆斯特丹大学）；Amsterdam Cardiovascular Sciences（阿姆斯特丹心血管科学研究所）；Informatics Institute, University of Amsterdam（阿姆斯特丹大学信息学研究所）
High-resolution medical images are beneficial for analysis but their acquisition may not always be feasible. Alternatively, high-resolution images can be created from low-resolution acquisitions using conventional upsampling methods, but such methods cannot exploit high-level contextual information contained in the images. Recently, better performing deep-learning based super-resolution methods have been introduced. However, these methods are limited by their supervised character, i.e. they require high-resolution examples for training. Instead, we propose an unsupervised deep learning semanti…
Head movement during scanning impedes activation detection in fMRI studies. Head motion in fMRI acquired using slice-based Echo Planar Imaging (EPI) can be estimated and compensated by aligning the images onto a reference volume through image registration. However, registering EPI images volume to volume fails to consider head motion between slices, which may lead to severely biased head motion estimates. Slice-to-volume registration can be used to estimate motion parameters for each slice by more accurately representing the image acquisition sequence. However, accurate slice to volume mapping…
Silesian University in Opava（奥帕瓦西里西亚大学）；University of Oxford（牛津大学）；Ostrava University（俄斯特拉发大学）
Matching MRI brain images between patients or mapping patients' MRI slices to the simulated atlas of a brain is key to the automatic registration of MRI of a brain. The ability to match MRI images would also enable such applications as indexing and searching MRI images among multiple patients or selecting images from the region of interest. In this work, we have introduced robustness, accuracy and cumulative distance metrics and methodology that allows us to compare different techniques and approaches in matching brain MRI of different patients or matching MRI brain slice to a position in the …
University of North Carolina at Chapel Hill（北卡罗来纳大学教堂山分校）；Korea University（高丽大学）
In this paper, we introduce an image quality assessment (IQA) method for pediatric T1- and T2-weighted MR images. IQA is first performed slice-wise using a nonlocal residual neural network (NR-Net) and then volume-wise by agglomerating the slice QA results using random forest. Our method requires only a small amount of quality-annotated images for training and is designed to be robust to annotation noise that might occur due to rater errors and the inevitable mix of good and bad slices in an image volume. Using a small set of quality-assessed images, we pre-train NR-Net to annotate each image …
MRI Plane Orientation Detection using a Context-Aware 2.5D Model
Department of Computer Science, University of Massachusetts Boston（计算机科学系，马萨诸塞大学波士顿分校）；Department of Physics, University of Massachusetts Boston（物理系，马萨诸塞大学波士顿分校）
University of Science and Technology Beijing（北京科技大学）；University of Central Florida（中佛罗里达大学）
For 3D medical image (e.g. CT and MRI) segmentation, the difficulty of segmenting each slice in a clinical case varies greatly. Previous research on volumetric medical image segmentation in a slice-by-slice manner conventionally use the identical 2D deep neural network to segment all the slices of the same case, ignoring the data heterogeneity among image slices. In this paper, we focus on multi-modal 3D MRI brain tumor segmentation and propose a dynamic architecture network named Med-DANet based on adaptive model selection to achieve effective accuracy and efficiency trade-off. For each slice…
Magnetic resonance (MR) images collected in 2D clinical protocols typically have large inter-slice spacing, resulting in high in-plane resolution and reduced through-plane resolution. Super-resolution technique can enhance the through-plane resolution of MR images to facilitate downstream visualization and computer-aided diagnosis. However, most existing works train the super-resolution network at a fixed scaling factor, which is not friendly to clinical scenes of varying inter-slice spacing in MR scanning. Inspired by the recent progress in implicit neural representation, we propose a Spatial…
