Phase-contrast magnetic resonance imaging (MRI) provides time-resolved quantification of blood flow dynamics that can aid clinical diagnosis. Long in vivo scan times due to repeated three-dimensional (3D) volume sampling over cardiac phases and breathing cycles necessitate accelerated imaging techniques that leverage data correlations. Standard compressed sensing reconstruction methods require tuning of hyperparameters and are computationally expensive, which diminishes the potential reduction of examination times. We propose an efficient model-based deep neural reconstruction network and evaluate its performance on clinical aortic flow data. The network is shown to reconstruct undersampled 4D flow MRI data in under a minute on standard consumer hardware. Remarkably, the relatively low amounts of tunable parameters allowed the network to be trained on images from 11 reference scans while generalizing well to retrospective and prospective undersampled data for various acceleration factors and anatomies.
Graz University of Technology（格拉茨技术大学）；Silicon Austria Labs（奥地利硅谷实验室）；University of Bonn（波恩大学）；Johannes Kepler University（约翰·开普勒大学）；Technical University of Munich（慕尼黑工业大学）；Imperial College London（伦敦帝国理工学院）；NYU School of Medicine（纽约大学医学院）
Recent deep learning approaches focus on improving quantitative scores of dedicated benchmarks, and therefore only reduce the observation-related (aleatoric) uncertainty. However, the model-immanent (epistemic) uncertainty is less frequently systematically analyzed. In this work, we introduce a Bayesian variational framework to quantify the epistemic uncertainty. To this end, we solve the linear inverse problem of undersampled MRI reconstruction in a variational setting. The associated energy functional is composed of a data fidelity term and the total deep variation (TDV) as a learned paramet…
University of Florida（佛罗里达大学）；Georgia State University（佐治亚州立大学）
Generating multi-contrasts/modal MRI of the same anatomy enriches diagnostic information but is limited in practice due to excessive data acquisition time. In this paper, we propose a novel deep-learning model for joint reconstruction and synthesis of multi-modal MRI using incomplete k-space data of several source modalities as inputs. The output of our model includes reconstructed images of the source modalities and high-quality image synthesized in the target modality. Our proposed model is formulated as a variational problem that leverages several learnable modality-specific feature extract…
Facebook AI Research (FAIR)（Facebook人工智能研究院（FAIR））；NYU School of Medicine（纽约大学医学院）
The slow acquisition speed of magnetic resonance imaging (MRI) has led to the development of two complementary methods: acquiring multiple views of the anatomy simultaneously (parallel imaging) and acquiring fewer samples than necessary for traditional signal processing methods (compressed sensing). While the combination of these methods has the potential to allow much faster scan times, reconstruction from such undersampled multi-coil data has remained an open problem. In this paper, we present a new approach to this problem that extends previously proposed variational methods by learning ful…
Automatic MRI brain tumor segmentation is of vital importance for the disease diagnosis, monitoring, and treatment planning. In this paper, we propose a two-stage encoder-decoder based model for brain tumor subregional segmentation. Variational autoencoder regularization is utilized in both stages to prevent the overfitting issue. The second-stage network adopts attention gates and is trained additionally using an expanded dataset formed by the first-stage outputs. On the BraTS 2020 validation dataset, the proposed method achieves the mean Dice score of 0.9041, 0.8350, and 0.7958, and Hausdorf…
University of Iowa（爱荷华大学）；Philips HealthCare（飞利浦医疗）；University of Wisconsin（威斯康星大学）；GE HealthCare（通用电气医疗）
Free-breathing cardiac MRI schemes are emerging as competitive alternatives to breath-held cine MRI protocols, enabling applicability to pediatric and other population groups that cannot hold their breath. Because the data from the slices are acquired sequentially, the cardiac/respiratory motion patterns may be different for each slice; current free-breathing approaches perform independent recovery of each slice. In addition to not being able to exploit the inter-slice redundancies, manual intervention or sophisticated post-processing methods are needed to align the images post-recovery for qu…
Netherlands Cancer Institute（荷兰癌症研究所）；qurAI group（qurAI团队）；University of Amsterdam（阿姆斯特丹大学）
Magnetic Resonance Imaging can produce detailed images of the anatomy and physiology of the human body that can assist doctors in diagnosing and treating pathologies such as tumours. However, MRI suffers from very long acquisition times that make it susceptible to patient motion artifacts and limit its potential to deliver dynamic treatments. Conventional approaches such as Parallel Imaging and Compressed Sensing allow for an increase in MRI acquisition speed by reconstructing MR images from sub-sampled MRI data acquired using multiple receiver coils. Recent advancements in Deep Learning combi…
Speech Audio Generation from dynamic MRI via a Knowledge Enhanced Conditional Variational Autoencoder
Department of Computer Science, The University of Hong Kong（香港大学计算机科学系）；School of Software Engineering, Xi'an Jiaotong University（西安交通大学软件工程学院）；Wake Forest University School of Medicine（威克森林大学医学学院）；Department of Radiology, Harvard Medical School（哈佛医学院放射科）
McGill University（麦吉尔大学）；MILA (Québec Artificial Intelligence Institute)（MILA（魁北克人工智能研究所））；University of Edinburgh（爱丁堡大学）；The Alan Turing Institute（艾伦·图灵研究所）
We propose a hierarchically structured variational inference model for accurately disentangling observable evidence of disease (e.g. brain lesions or atrophy) from subject-specific anatomy in brain MRIs. With flexible, partially autoregressive priors, our model (1) addresses the subtle and fine-grained dependencies that typically exist between anatomical and pathological generating factors of an MRI to ensure the clinical validity of generated samples; (2) preserves and disentangles finer pathological details pertaining to a patient's disease state. Additionally, we experiment with an alternat…
Inspired by the first-order method of Malitsky and Pock, we propose a new variational framework for compressed MR image reconstruction which introduces the application of a rotation-invariant discretization of total variation functional into MR imaging while exploiting BM3D frame as a sparsifying transform. In the first step, we provide theoretical and numerical analysis establishing the exceptional rotation-invariance property of this total variation functional and observe its superiority over other well-known variational regularization terms in both upright and rotated imaging setups. Thereu…
Interpretable Representation Learning of Cardiac MRI via Attribute Regularization
Institute of Machine Learning in Biomedical Imaging, Helmholtz Munich（亥姆霍兹慕尼黑生物医学成像机器学习研究所）；Helmholtz AI, Helmholtz Munich（亥姆霍兹慕尼黑亥姆霍兹人工智能研究所）；School of Computation, Information and Technology, Technical University of Munich（慕尼黑工业大学计算、信息与技术学院）；School of Biomedical Engineering and Imaging Sciences, King’s College London（伦敦国王学院生物医学工程与影像科学学院）
Otto von Guericke University Magdeburg（马格德堡奥托·冯·格里克大学）；Center for Behavioral Brain Sciences（行为脑科学研究中心）；German Center for Neurodegenerative Disease（德国神经退行性疾病研究中心）
Expert interpretation of anatomical images of the human brain is the central part of neuro-radiology. Several machine learning-based techniques have been proposed to assist in the analysis process. However, the ML models typically need to be trained to perform a specific task, e.g., brain tumour segmentation or classification. Not only do the corresponding training data require laborious manual annotations, but a wide variety of abnormalities can be present in a human brain MRI - even more than one simultaneously, which renders representation of all possible anomalies very challenging. Hence, …
XLSTM-HVED: Cross-Modal Brain Tumor Segmentation and MRI Reconstruction Method Using Vision XLSTM and Heteromodal Variational Encoder-Decoder
Hangzhou Dianzi University（杭州电子科技大学）；Shenzhen Research Institute of Big Data（深圳大数据研究院）；Children's Hospital, Zhejiang University School of Medicine（浙江大学医学院附属儿童医院）；Sino-Finland Joint AI Laboratory for Child Health of Zhejiang Province（浙江省中芬儿童健康人工智能联合实验室）
Autism Spectrum Disorder Classification with Interpretability in Children based on Structural MRI Features Extracted using Contrastive Variational Autoencoder
Shenzhen Institute of Advanced Technology, Chinese Academy of Sciences（中国科学院深圳先进技术研究院）；University of Chinese Academy of Sciences（中国科学院大学）；Shenzhen Key Laboratory of Intelligent Bioinformatics（深圳市智能生物信息学重点实验室）；Center for High Performance Computing, Shenzhen Institute of Advanced Technology, Chinese Academy of Sciences（中国科学院深圳先进技术研究院高性能计算中心）
MRI-to-CT Synthesis With Cranial Suture Segmentations Using A Variational Autoencoder Framework
Sheikh Zayed Institute for Pediatric Surgical Innovation, Children’s National Hospital, Washington, DC, USA（Sheikh Zayed儿童外科创新研究所，Children’s National医院，华盛顿特区，美国）；CIBORG Lab, Department of Radiology, Children’s Hospital Los Angeles, Los Angeles, CA, USA（CIBORG实验室，放射科，Children’s Hospital Los Angeles，洛杉矶，加利福尼亚州，美国）；Departments of Radiology and Biomedical Engineering, University of Southern California, Los Angeles, CA, USA（放射科和生物医学工程系，美国南加州大学，洛杉矶，加利福尼亚州，美国）；School of Medicine and Health Sciences, George Washington University, Washington, DC, USA（医学与健康科学学院，乔治华盛顿大学，华盛顿特区，美国）
University of Sussex（萨塞克斯大学）；Brighton and Sussex Medical School（布莱顿和萨塞克斯医学院）；St. Vincent’s University Hospital（圣文森特大学医院）；University of Nottingham（诺丁汉大学）；Cardiff University（卡迪夫大学）
Streamlined qBOLD acquisitions enable experimentally straightforward observations of brain oxygen metabolism. $R_2^\prime$ maps are easily inferred; however, the Oxygen extraction fraction (OEF) and deoxygenated blood volume (DBV) are more ambiguously determined from the data. As such, existing inference methods tend to yield very noisy and underestimated OEF maps, while overestimating DBV. This work describes a novel probabilistic machine learning approach that can infer plausible distributions of OEF and DBV. Initially, we create a model that produces informative voxelwise prior distribution…
Purpose: To develop a synergistic image reconstruction framework that exploits multicontrast (MC), multicoil, and compressed sensing (CS) redundancies in magnetic resonance imaging (MRI). Approach: CS, MC acquisition, and parallel imaging (PI) have been individually well developed, but the combination of the three has not been equally well studied, much less the potential benefits of isotropy within such a setting. Inspired by total variation theory, we introduce an isotropic MC image regularizer and attain its full potential by integrating it into compressed MC multicoil MRI. A convex optimiz…
Automatic segmentation of brain abnormalities is challenging, as they vary considerably from one pathology to another. Current methods are supervised and require numerous annotated images for each pathology, a strenuous task. To tackle anatomical variability, Unsupervised Anomaly Detection (UAD) methods are proposed, detecting anomalies as outliers of a healthy model learned using a Variational Autoencoder (VAE). Previous work on UAD adopted a 2D approach, meaning that MRIs are processed as a collection of independent slices. Yet, it does not fully exploit the spatial information contained in …
Following the success of deep learning in a wide range of applications, neural network-based machine-learning techniques have received significant interest for accelerating magnetic resonance imaging (MRI) acquisition and reconstruction strategies. A number of ideas inspired by deep learning techniques for computer vision and image processing have been successfully applied to nonlinear image reconstruction in the spirit of compressed sensing for accelerated MRI. Given the rapidly growing nature of the field, it is imperative to consolidate and summarize the large number of deep learning method…
Weakly Convex Ridge Regularization for 3D Non-Cartesian MRI Reconstruction
Faculty of Mathematics, Chemnitz University of Technology（开姆尼茨工业大学数学学院）；NeuroSpin, Frédéric Joliot Institute for Life Sciences, CEA Paris-Saclay（法国原子能委员会巴黎-萨克雷大学弗雷德里克·约里奥生命科学研究所神经自旋中心）；MIND, Inria Saclay Centre（法国国家信息与自动化研究所萨克雷中心MIND团队）；Siemens Healthineers（西门子医疗）
Department of Radiology, Massachusetts General Hospital（麻省总医院放射科）；Radiology, Perelman School of Medicine, University of Pennsylvania（宾夕法尼亚大学佩雷尔曼医学院放射科）
In fetal-brain MRI, head-pose changes between prescription and acquisition present a challenge to obtaining the standard sagittal, coronal and axial views essential to clinical assessment. As motion limits acquisitions to thick slices that preclude retrospective resampling, technologists repeat ~55-second stack-of-slices scans (HASTE) with incrementally reoriented field of view numerous times, deducing the head pose from previous stacks. To address this inefficient workflow, we propose a robust head-pose detection algorithm using full-uterus scout scans (EPI) which take ~5 seconds to acquire. …
Reliable MRI is crucial for accurate interpretation in therapeutic and diagnostic tasks. However, undersampling during MRI acquisition as well as the overparameterized and non-transparent nature of deep learning (DL) leaves substantial uncertainty about the accuracy of DL reconstruction. With this in mind, this study aims to quantify the uncertainty in image recovery with DL models. To this end, we first leverage variational autoencoders (VAEs) to develop a probabilistic reconstruction scheme that maps out (low-quality) short scans with aliasing artifacts to the diagnostic-quality ones. The VA…
Deep learning methods for classifying medical images have demonstrated impressive accuracy in a wide range of tasks but often these models are hard to interpret, limiting their applicability in clinical practice. In this work we introduce a convolutional neural network model for identifying disease in temporal sequences of cardiac MR segmentations which is interpretable in terms of clinically familiar measurements. The model is based around a variational autoencoder, reducing the input into a low-dimensional latent space in which classification occurs. We then use the recently developed `conce…
École polytechnique fédérale de Lausanne (EPFL)（洛桑联邦理工学院）；University Hospital (CHUV)（沃州大学医院）；University of Lausanne (UNIL)（洛桑大学）；Center for Biomedical Imaging (CIBM)（生物医学成像中心）
We propose a novel unsupervised deep-learning-based algorithm for dynamic magnetic resonance imaging (MRI) reconstruction. Dynamic MRI requires rapid data acquisition for the study of moving organs such as the heart. Existing reconstruction methods suffer from restrictions either in the model design or in the absence of ground-truth data, resulting in low image quality. We introduce a generalized version of the deep-image-prior approach, which optimizes the network weights to fit a sequence of sparsely acquired dynamic MRI measurements. Our method needs neither prior training nor additional da…
Fast data acquisition in Magnetic Resonance Imaging (MRI) is vastly in demand and scan time directly depends on the number of acquired k-space samples. The data-driven methods based on deep neural networks have resulted in promising improvements, compared to the conventional methods, in image reconstruction algorithms. The connection between deep neural network and Ordinary Differential Equation (ODE) has been observed and studied recently. The studies show that different residual networks can be interpreted as Euler discretization of an ODE. In this paper, we propose an ODE-based deep network…
Harvard Medical School（哈佛医学院）；Boston Children’s Hospital（波士顿儿童医院）
Fast data acquisition in Magnetic Resonance Imaging (MRI) is vastly in demand and scan time directly depends on the number of acquired k-space samples. Conventional MRI reconstruction methods for fast MRI acquisition mostly relied on different regularizers which represent analytical models of sparsity. However, recent data-driven methods based on deep learning has resulted in promising improvements in image reconstruction algorithms. In this paper, we propose a deep plug-and-play prior framework for parallel MRI reconstruction problems which utilize a deep neural network (DNN) as an advanced d…
University of Amsterdam（阿姆斯特丹大学）；Facebook AI Research（Facebook人工智能研究院）
Most current approaches to undersampled multi-coil MRI reconstruction focus on learning the reconstruction model for a fixed, equidistant acquisition trajectory. In this paper, we study the problem of joint learning of the reconstruction model together with acquisition policies. To this end, we extend the End-to-End Variational Network with learnable acquisition policies that can adapt to different data points. We validate our model on a coil-compressed version of the large scale undersampled multi-coil fastMRI dataset using two undersampling factors: $4\times$ and $8\times$. Our experiments s…
JSS Academy of Higher Education and Research（JSS高等教育与研究学院）；University of Delhi（德里大学）
Early detection of Alzheimer disease is crucial for deploying interventions and slowing the disease progression. A lot of machine learning and deep learning algorithms have been explored in the past decade with the aim of building an automated detection for Alzheimer. Advancements in data augmentation techniques and advanced deep learning architectures have opened up new frontiers in this field, and research is moving at a rapid speed. Hence, the purpose of this survey is to provide an overview of recent research on deep learning models for Alzheimer disease diagnosis. In addition to categoriz…
Memorial Sloan Kettering Cancer Center（纪念斯隆-凯特琳癌症中心）
Our contribution is a unified cross-modality feature disentagling approach for multi-domain image translation and multiple organ segmentation. Using CT as the labeled source domain, our approach learns to segment multi-modal (T1-weighted and T2-weighted) MRI having no labeled data. Our approach uses a variational auto-encoder (VAE) to disentangle the image content from style. The VAE constrains the style feature encoding to match a universal prior (Gaussian) that is assumed to span the styles of all the source and target modalities. The extracted image style is converted into a latent style sc…
Institute of Medical Technology and Intelligent Systems, Hamburg University of Technology（汉堡工业大学医学技术与智能系统研究所）；jung diagnostics GmbH（jung diagnostics 有限公司）
Purpose. Brain Magnetic Resonance Images (MRIs) are essential for the diagnosis of neurological diseases. Recently, deep learning methods for unsupervised anomaly detection (UAD) have been proposed for the analysis of brain MRI. These methods rely on healthy brain MRIs and eliminate the requirement of pixel-wise annotated data compared to supervised deep learning. While a wide range of methods for UAD have been proposed, these methods are mostly 2D and only learn from MRI slices, disregarding that brain lesions are inherently 3D and the spatial context of MRI volumes remains unexploited. Metho…
MIT（麻省理工学院）；Tsinghua University（清华大学）；Boston Children’s Hospital（波士顿儿童医院）；Harvard Medical School（哈佛医学院）
The performance and diagnostic utility of magnetic resonance imaging (MRI) in pregnancy is fundamentally constrained by fetal motion. Motion of the fetus, which is unpredictable and rapid on the scale of conventional imaging times, limits the set of viable acquisition techniques to single-shot imaging with severe compromises in signal-to-noise ratio and diagnostic contrast, and frequently results in unacceptable image quality. Surprisingly little is known about the characteristics of fetal motion during MRI and here we propose and demonstrate methods that exploit a growing repository of MRI ob…
University of Sherbrooke（舍布鲁克大学）；Université de Bourgogne Franche-Comté（勃艮第-弗朗什孔泰大学）；CArdiac Simulation & Imaging Software（心脏模拟与成像软件公司）；Université de Lyon（里昂大学）
Recent publications have shown that the segmentation accuracy of modern-day convolutional neural networks (CNN) applied on cardiac MRI can reach the inter-expert variability, a great achievement in this area of research. However, despite these successes, CNNs still produce anatomically inaccurate segmentations as they provide no guarantee on the anatomical plausibility of their outcome, even when using a shape prior. In this paper, we propose a cardiac MRI segmentation method which always produces anatomically plausible results. At the core of the method is an adversarial variational autoencod…
There is much recent interest in techniques to accelerate the data acquisition process in MRI by acquiring limited measurements. Often sophisticated reconstruction algorithms are deployed to maintain high image quality in such settings. In this work, we propose a data-driven sampler using a convolutional neural network, MNet, to provide object-specific sampling patterns adaptive to each scanned object. The network observes very limited low-frequency k-space data for each object and rapidly predicts the desired undersampling pattern in one go that achieves high image reconstruction quality. We …
L-TGVN: Leveraging Longitudinal Priors for Personalized Rapid MRI
NYU Center for Data Science（纽约大学数据科学中心）；Center for Advanced Imaging Innovation and Research (CAI²R)（先进成像创新与研究中心）；Courant Institute of Mathematical Sciences（数学科学学院）；Function Health
Cornell University（康奈尔大学）；Massachusetts Institute of Technology（麻省理工学院）；Massachusetts General Hospital（麻省总医院）；Harvard Medical School（哈佛医学院）
Reconstructing under-sampled k-space measurements in Compressed Sensing MRI (CS-MRI) is classically solved with regularized least-squares. Recently, deep learning has been used to amortize this optimization by training reconstruction networks on a dataset of under-sampled measurements. Here, a crucial design choice is the regularization function(s) and corresponding weight(s). In this paper, we explore a novel strategy of using a hypernetwork to generate the parameters of a separate reconstruction network as a function of the regularization weight(s), resulting in a regularization-agnostic rec…
Image2Flow: A hybrid image and graph convolutional neural network for rapid patient-specific pulmonary artery segmentation and CFD flow field calculation from 3D cardiac MRI data
Institute of Cardiovascular Science, University College London（伦敦大学学院心血管科学研究所）
Probabilistic modelling has been an essential tool in medical image analysis, especially for analyzing brain Magnetic Resonance Images (MRI). Recent deep learning techniques for estimating high-dimensional distributions, in particular Variational Autoencoders (VAEs), opened up new avenues for probabilistic modeling. Modelling of volumetric data has remained a challenge, however, because constraints on available computation and training data make it difficult effectively leverage VAEs, which are well-developed for 2D images. We propose a method to model 3D MR brain volumes distribution by combi…
Computational Radiology Laboratory, Boston Children’s Hospital（波士顿儿童医院计算放射实验室）；Harvard Medical School（哈佛医学院）
Fast data acquisition in Magnetic Resonance Imaging (MRI) is vastly in demand and scan time directly depends on the number of acquired k-space samples. Recently, the deep learning-based MRI reconstruction techniques were suggested to accelerate MR image acquisition. The most common issues in any deep learning-based MRI reconstruction approaches are generalizability and transferability. For different MRI scanner configurations using these approaches, the network must be trained from scratch every time with new training dataset, acquired under new configurations, to be able to provide good recon…
Institute of Machine Learning in Biomedical Imaging, Helmholtz Munich（亥姆霍兹慕尼黑生物医学成像机器学习研究所）；Helmholtz AI, Helmholtz Munich（亥姆霍兹慕尼黑人工智能中心）；School of Computation, Information and Technology, Technical University of Munich（慕尼黑工业大学计算、信息与技术学院）；School of Biomedical Engineering and Imaging Sciences, King's College London（伦敦国王学院生物医学工程与影像科学学院）
MRI Parameter Mapping via Gaussian Mixture VAE: Breaking the Assumption of Independent Pixels
UCL（伦敦大学学院）；UCL Hawkes Institute（伦敦大学学院霍克斯研究所）；UCL Satsuma Lab（伦敦大学学院萨摩亚实验室）；UCL Department of Medical Physics and Biomedical Engineering（伦敦大学学院医学物理与生物医学工程系）；UCL Medical School（伦敦大学学院医学院）；UCL Respiratory（伦敦大学学院呼吸科）；UCL Department of Computer Science（伦敦大学学院计算机科学系）；Cardiff University（卡迪夫大学）；Cardiff University Brain Research Imaging Center（卡迪夫大学脑研究成像中心）
Facebook AI Research（脸书人工智能研究院）；NYU School of Medicine（纽约大学医学院）；NYU Center for Data Science（纽约大学数据科学中心）；University of Florida（佛罗里达大学）
Accelerating Magnetic Resonance Imaging (MRI) by taking fewer measurements has the potential to reduce medical costs, minimize stress to patients and make MRI possible in applications where it is currently prohibitively slow or expensive. We introduce the fastMRI dataset, a large-scale collection of both raw MR measurements and clinical MR images, that can be used for training and evaluation of machine-learning approaches to MR image reconstruction. By introducing standardized evaluation criteria and a freely-accessible dataset, our goal is to help the community make rapid advances in the stat…
