In-scanner motion degrades the quality of magnetic resonance imaging (MRI) thereby reducing its utility in the detection of clinically relevant abnormalities. We introduce a deep learning-based MRI artifact reduction model (DMAR) to localize and correct head motion artifacts in brain MRI scans. Our approach integrates the latest advances in object detection and noise reduction in Computer Vision. Specifically, DMAR employs a two-stage approach: in the first, degraded regions are detected using the Single Shot Multibox Detector (SSD), and in the second, the artifacts within the found regions are reduced using a convolutional autoencoder (CAE). We further introduce a set of novel data augmentation techniques to address the high dimensionality of MRI images and the scarcity of available data. As a result, our model was trained on a large synthetic dataset of 225,000 images generated from 375 whole brain T1-weighted MRI scans. DMAR visibly reduces image artifacts when applied to both synthetic test images and 55 real-world motion-affected slices from 18 subjects from the multi-center Autism Brain Imaging Data Exchange (ABIDE) study. Quantitatively, depending on the level of degradation, our model achieves a 27.8%-48.1% reduction in RMSE and a 2.88--5.79 dB gain in PSNR on a 5000-sample set of synthetic images. For real-world artifact-affected scans from ABIDE, our model reduced the variance of image voxel intensity within artifact-affected brain regions (p = 0.014).
Retrospective artifact correction (RAC) improves image quality post acquisition and enhances image usability. Recent machine learning driven techniques for RAC are predominantly based on supervised learning and therefore practical utility can be limited as data with paired artifact-free and artifact-corrupted images are typically insufficient or even non-existent. Here we show that unwanted image artifacts can be disentangled and removed from an image via an RAC neural network learned with unpaired data. This implies that our method does not require matching artifact-corrupted data to be eithe…
The k-space data generated from magnetic resonance imaging (MRI) is only a finite sampling of underlying signals. Therefore, MRI images often suffer from low spatial resolution and Gibbs ringing artifacts. Previous studies tackled these two problems separately, where super resolution methods tend to enhance Gibbs artifacts, whereas Gibbs ringing removal methods tend to blur the images. It is also a challenge that high resolution ground truth is hard to obtain in clinical MRI. In this paper, we propose an unsupervised learning framework for both MRI super resolution and Gibbs artifacts removal …
Korea Advanced Institute of Science and Technology (KAIST)（韩国科学技术院）；Seoul National University College of Medicine（首尔大学医学院）
Diffusion-weighted MRI is nowadays performed routinely due to its prognostic ability, yet the quality of the scans are often unsatisfactory which can subsequently hamper the clinical utility. To overcome the limitations, here we propose a fully unsupervised quality enhancement scheme, which boosts the resolution and removes the motion artifact simultaneously. This process is done by first training the network using optimal transport driven cycleGAN with stochastic degradation block which learns to remove aliasing artifacts and enhance the resolution, then using the trained network in the test …
3D Brain MRI Classification for Alzheimer Diagnosis Using CNN with Data Augmentation
Vo Thien Nhan Institute of Engineering, Ho Chi Minh City University of Technology (HUTECH), Vietnam（越南胡志明市技术大学（HUTECH）工程学院）；Ho Bac Nam Faculty of Computer Science and Engineering, Ho Chi Minh City University of Technology (HCMUT), Vietnam National University Ho Chi Minh City (VNU-HCM), Vietnam（越南胡志明市技术大学（HCMUT）计算机科学与工程学院，越南胡志明市国家大学（VNU-HCM））
For several years, numerous attempts have been made to reduce noise and artifacts in MRI. Although there have been many successful methods to address these problems, practical implementation for clinical images is still challenging because of its complicated mechanism. Recently, deep learning received considerable attention, emerging as a machine learning approach in delivering robust MR image processing. The purpose here is therefore to explore further and review noise and artifact reduction using deep learning for MRI.
School of Electrical Engineering, Korea Advanced Institute of Science and Technology (KAIST)（韩国科学技术院电气工程学院）
An attention guided scheme for metal artifact correction in MRI using deep neural network is proposed in this paper. The inputs of the networks are two distorted images obtained with dual-polarity readout gradients. With MR image generation module and the additional data consistency loss to the previous work [1], the network is trained to estimate the frequency-shift map, off-resonance map, and attention map. The attention map helps to produce better distortion-corrected images by weighting on more relevant distortion-corrected images where two distortion-corrected images are produced with hal…
Technical University of Denmark（丹麦技术大学）；icometrix（icometrix公司）；Technical University of Munich（慕尼黑工业大学）；University of Antwerp（安特卫普大学）；Harvard Medical School（哈佛医学院）
Most publicly available brain MRI datasets are very homogeneous in terms of scanner and protocols, and it is difficult for models that learn from such data to generalize to multi-center and multi-scanner data. We propose a novel data augmentation approach with the aim of approximating the variability in terms of intensities and contrasts present in real world clinical data. We use a Gaussian Mixture Model based approach to change tissue intensities individually, producing new contrasts while preserving anatomical information. We train a deep learning model on a single scanner dataset and evalu…
Rensselaer Polytechnic Institute（伦斯勒理工学院）；Cedars-Sinai Medical Center（西达赛奈医疗中心）；University of California, Los Angeles（加州大学洛杉矶分校）
Cine cardiac magnetic resonance imaging (MRI) is widely used for diagnosis of cardiac diseases thanks to its ability to present cardiovascular features in excellent contrast. As compared to computed tomography (CT), MRI, however, requires a long scan time, which inevitably induces motion artifacts and causes patients' discomfort. Thus, there has been a strong clinical motivation to develop techniques to reduce both the scan time and motion artifacts. Given its successful applications in other medical imaging tasks such as MRI super-resolution and CT metal artifact reduction, deep learning is a…
Philips Healthcare（飞利浦医疗）；University of Washington（华盛顿大学）
Patient motion during the magnetic resonance imaging (MRI) acquisition process results in motion artifacts, which limits the ability of radiologists to provide a quantitative assessment of a condition visualized. Often times, radiologists either "see through" the artifacts with reduced diagnostic confidence, or the MR scans are rejected and patients are asked to be recalled and re-scanned. Presently, there are many published approaches that focus on MRI artifact detection and correction. However, the key question of the bias exhibited by these algorithms on motion corrupted MRI images is still…
University of Southern California（南加州大学）；Technical University of Munich（慕尼黑工业大学）；Rice University（莱斯大学）
Deep neural networks have emerged as very successful tools for image restoration and reconstruction tasks. These networks are often trained end-to-end to directly reconstruct an image from a noisy or corrupted measurement of that image. To achieve state-of-the-art performance, training on large and diverse sets of images is considered critical. However, it is often difficult and/or expensive to collect large amounts of training images. Inspired by the success of Data Augmentation (DA) for classification problems, in this paper, we propose a pipeline for data augmentation for accelerated MRI re…
Mask-guided Data Augmentation for Multiparametric MRI Generation with a Rare Hepatocellular Carcinoma
Universidad Industrial de Santander（桑坦德工业大学）；University of Toulouse（图卢兹大学）；Université Paul Sabatier Toulouse 3（图卢兹第三大学（保罗·萨巴蒂埃大学））；Bicêtre University Hospital（比塞特尔大学医院）；Assistance publique des hôpitaux de Paris（巴黎公共援助医院）；Université Claude Bernard Lyon 1（里昂第一大学（克洛德·贝尔纳里昂第一大学））
Yonsei University（延世大学）；GE Healthcare（通用电气医疗）；Seoul National University Hospital（首尔大学医院）
In this paper, we develop an efficient retrospective deep learning method called stacked U-Nets with self-assisted priors to address the problem of rigid motion artifacts in MRI. The proposed work exploits the usage of additional knowledge priors from the corrupted images themselves without the need for additional contrast data. The proposed network learns missed structural details through sharing auxiliary information from the contiguous slices of the same distorted subject. We further design a refinement stacked U-Nets that facilitates preserving of the image spatial details and hence improv…
Local Gamma Augmentation for Ischemic Stroke Lesion Segmentation on MRI
University of Copenhagen（哥本哈根大学）；Cerebriu A/S（Cerebriu公司）；Rigshospitalet, Copenhagen, Denmark（哥本哈根大学医院）；Pioneer Centre for Artificial Intelligence, University of Copenhagen（哥本哈根大学人工智能先锋中心）
Artifacts are a common occurrence in Diffusion MRI (dMRI) scans. Identifying and removing them is essential to ensure the accuracy and viability of any post processing carried out on these scans. This makes QC (quality control) a crucial first step prior to any analysis of dMRI data. Several QC methods for artifact detection exist, however they suffer from problems like requiring manual intervention and the inability to generalize across different artifacts and datasets. In this paper, we propose an automated deep learning (DL) pipeline that utilizes a 3D-Densenet architecture to train a model…
Parallel imaging is a widely-used technique to accelerate magnetic resonance imaging (MRI). However, current methods still perform poorly in reconstructing artifact-free MRI images from highly undersampled k-space data. Recently, implicit neural representation (INR) has emerged as a new deep learning paradigm for learning the internal continuity of an object. In this study, we adopted INR to parallel MRI reconstruction. The MRI image was modeled as a continuous function of spatial coordinates. This function was parameterized by a neural network and learned directly from the measured k-space it…
KTH Royal Institute of Technology（KTH皇家理工学院）；Science for Life Laboratory（生命科学实验室）
Artifacts on magnetic resonance scans are a serious challenge for both radiologists and computer-aided diagnosis systems. Most commonly, artifacts are caused by motion of the patients, but can also arise from device-specific abnormalities such as noise patterns. Irrespective of the source, artifacts can not only render a scan useless, but can potentially induce misdiagnoses if left unnoticed. For instance, an artifact may masquerade as a tumor or other abnormality. Retrospective artifact correction (RAC) is concerned with removing artifacts after the scan has already been taken. In this work, …
The volume estimation of brain regions from MRI data is a key problem in many clinical applications, where the acquisition of data at high spatial resolution is desirable. While parallel MRI and constrained image reconstruction algorithms can accelerate the scans, image reconstruction artifacts are inevitable, especially at high acceleration factors. We introduce a novel image domain deep-learning framework for calibrationless parallel MRI reconstruction, coupled with a segmentation network to improve image quality and to reduce the vulnerability of current segmentation algorithms to image art…
Motion artifacts are a common occurrence in the Magnetic Resonance Imaging (MRI) exam. Motion during acquisition has a profound impact on workflow efficiency, often requiring a repeat of sequences. Furthermore, motion artifacts may escape notice by technologists, only to be revealed at the time of reading by the radiologists, affecting their diagnostic quality. Designing a computer-aided tool for automatic motion detection and elimination can improve the diagnosis, however, it needs a deep understanding of motion characteristics. Motion artifacts in MRI have a complex nature and it is directly…
We develop and evaluate a neural network-based method for Gibbs artifact and noise removal. A convolutional neural network (CNN) was designed for artifact removal in diffusion-weighted imaging data. Two implementations were considered: one for magnitude images and one for complex images. Both models were based on the same encoder-decoder structure and were trained by simulating MRI acquisitions on synthetic non-MRI images. Both machine learning methods were able to mitigate artifacts in diffusion-weighted images and diffusion parameter maps. The CNN for complex images was also able to reduce a…
Whole gland (WG) segmentation of the prostate plays a crucial role in detection, staging and treatment planning of prostate cancer (PCa). Despite promise shown by deep learning (DL) methods, they rely on the availability of a considerable amount of annotated data. Augmentation techniques such as translation and rotation of images present an alternative to increase data availability. Nevertheless, the amount of information provided by the transformed data is limited due to the correlation between the generated data and the original. Based on the recent success of generative adversarial networks…
Sun Yat-sen University（中山大学）；The Chinese University of Hong Kong（香港中文大学）；Royal Brompton Hospital（皇家布朗普顿医院）；Imperial College London（伦敦帝国理工学院）
The long acquisition time has limited the accessibility of magnetic resonance imaging (MRI) because it leads to patient discomfort and motion artifacts. Although several MRI techniques have been proposed to reduce the acquisition time, compressed sensing in magnetic resonance imaging (CS-MRI) enables fast acquisition without compromising SNR and resolution. However, existing CS-MRI methods suffer from the challenge of aliasing artifacts. This challenge results in the noise-like textures and missing the fine details, thus leading to unsatisfactory reconstruction performance. To tackle this chal…
Deep neural networks have enabled improved image quality and fast inference times for various inverse problems, including accelerated magnetic resonance imaging (MRI) reconstruction. However, such models require a large number of fully-sampled ground truth datasets, which are difficult to curate, and are sensitive to distribution drifts. In this work, we propose applying physics-driven data augmentations for consistency training that leverage our domain knowledge of the forward MRI data acquisition process and MRI physics to achieve improved label efficiency and robustness to clinically-releva…
Counterfactual MRI Data Augmentation using Conditional Denoising Diffusion Generative Models
Instituto Superior Técnico（高等技术学院）；Universidade de Lisboa（里斯本大学）；Digital Surgery LAB（数字外科实验室）；Champalimaud Foundation（尚帕利莫基金会）；Faculdade de Medicina, Universidade de Lisboa（里斯本大学医学院）；Instituto de Telecomunicações（电信研究所）
In clinical practice MR images are often first seen by radiologists long after the scan. If image quality is inadequate either patients have to return for an additional scan, or a suboptimal interpretation is rendered. An automatic image quality assessment (IQA) would enable real-time remediation. Existing IQA works for MRI give only a general quality score, agnostic to the cause of and solution to low-quality scans. Furthermore, radiologists' image quality requirements vary with the scan type and diagnostic task. Therefore, the same score may have different implications for different scans. W…
University of Toronto（多伦多大学）；Sunnybrook Research Institute（桑尼布鲁克研究所）
The segmentation of the breast from the chest wall is an important first step in the analysis of breast magnetic resonance images. 3D U-nets have been shown to obtain high segmentation accuracy and appear to generalize well when trained on one scanner type and tested on another scanner, provided that a very similar T1-weighted MR protocol is used. There has, however, been little work addressing the problem of domain adaptation when image intensities or patient orientation differ markedly between the training set and an unseen test set. To overcome the domain shift we propose to apply extensive…
MIT（麻省理工学院）；Boston Children’s Hospital（波士顿儿童医院）；Harvard Medical School（哈佛医学院）
Fetal MRI is heavily constrained by unpredictable and substantial fetal motion that causes image artifacts and limits the set of viable diagnostic image contrasts. Current mitigation of motion artifacts is predominantly performed by fast, single-shot MRI and retrospective motion correction. Estimation of fetal pose in real time during MRI stands to benefit prospective methods to detect and mitigate fetal motion artifacts where inferred fetal motion is combined with online slice prescription with low-latency decision making. Current developments of deep reinforcement learning (DRL), offer a nov…
University of Maryland School of Medicine（马里兰大学医学院）；Pacific Northwest National Laboratory（太平洋西北国家实验室）；University of Maryland, College Park（马里兰大学帕克分校）
Purpose To develop and evaluate a deep learning-based method (MC-Net) to suppress motion artifacts in brain magnetic resonance imaging (MRI). Methods MC-Net was derived from a UNet combined with a two-stage multi-loss function. T1-weighted axial brain images contaminated with synthetic motions were used to train the network. Evaluation used simulated T1 and T2-weighted axial, coronal, and sagittal images unseen during training, as well as T1-weighted images with motion artifacts from real scans. Performance indices included the peak signal to noise ratio (PSNR), structural similarity index mea…
The application of deep learning to build accurate predictive models from functional neuroimaging data is often hindered by limited dataset sizes. Though data augmentation can help mitigate such training obstacles, most data augmentation methods have been developed for natural images as in computer vision tasks such as CIFAR, not for medical images. This work helps to fills in this gap by proposing a method for generating new functional Magnetic Resonance Images (fMRI) with realistic brain morphology. This method is tested on a challenging task of predicting antidepressant treatment response f…
Anatomically Faithful Artifact Suppression in SENSE Accelerated Brain MRI
East China Normal University（华东师范大学）；Affiliated Hospital of Guizhou Medical University（贵州医科大学附属医院）；West China Hospital, Sichuan University（四川大学华西医院）；Chinese Academy of Medical Sciences（中国医学科学院）；RadioDynamic Medical（镭动医疗）；Lucile Packard Children’s Hospital（露西尔·帕卡德儿童医院）
Improving Factuality of 3D Brain MRI Report Generation with Paired Image-domain Retrieval and Text-domain Augmentation
Cancer Biology, Seoul National University College of Medicine, Korea（首尔国立大学医学院癌症生物学系，韩国）；Radiology, Massachusetts General Hospital（麻省总医院放射科）；Harvard Medical School（哈佛医学院）；Biomedical Systems Informatics, Yonsei University College of Medicine, Korea（延世大学医学院生物医学系统信息学系，韩国）；Graduate School, Yonsei University, Korea（延世大学研究生院，韩国）；Radiology, Seoul National University College of Medicine, Korea（首尔国立大学医学院放射科，韩国）；Radiology, Seoul National University Hospital, Korea（首尔国立大学医院放射科，韩国）；Radiology, Seoul National University Bundang Hospital, Korea（首尔国立大学 Bundang 医院放射科，韩国）；Radiology, SMG-SNU Boramae Medical Center, Korea（SMG-SNU Boramae 医疗中心放射科，韩国）；Psychiatry, Yonsei University College of Medicine, Korea（延世大学医学院精神病学系，韩国）；Behavioral Sciences in Medicine, Yonsei University College of Medicine, Korea（延世大学医学院医学行为科学系，韩国）；Yonsei Institute for Digital Health, Korea（延世大学数字健康研究院，韩国）；Kim Jaechul Graduate School of AI, KAIST, Korea（金 Jaechul人工智能研究生院，韩国）
Reconstruction of Cardiac Cine MRI Using Motion-Guided Deformable Alignment and Multi-Resolution Fusion
School of Health Science and Engineering, University of Shanghai for Science and Technology（上海理工大学健康科学与工程学院）；ToolSensing Technologies Co., Ltd（拓深科技有限公司）；School of Medical Instruments, Shanghai University of Medicine and Health Sciences（上海健康医学院医疗器械学院）；Shanghai Key Laboratory of Multidimensional Information Processing, School of Communication & Electronic Engineering, East China Normal University（华东师范大学通信与电子工程学院多维信息处理上海重点实验室）
Image Sciences Institute, University Medical Center Utrecht（乌得勒支大学医学中心图像科学研究所）；Wilhelmina Children’s Hospital, University Medical Center Utrecht（乌得勒支大学医学中心威廉明娜儿童医院）；Brain Center Rudolf Magnus, University Medical Center Utrecht（乌得勒支大学医学中心鲁道夫·马格努斯脑中心）
Automatic neonatal brain tissue segmentation in preterm born infants is a prerequisite for evaluation of brain development. However, automatic segmentation is often hampered by motion artifacts caused by infant head movements during image acquisition. Methods have been developed to remove or minimize these artifacts during image reconstruction using frequency domain data. However, frequency domain data might not always be available. Hence, in this study we propose a method for removing motion artifacts from the already reconstructed MR scans. The method employs a generative adversarial network…
In magnetic resonance imaging (MRI), inhomogeneity in the main magnetic field used for imaging, referred to as off-resonance, can lead to image artifacts ranging from mild to severe depending on the application. Off-resonance artifacts, such as signal loss, geometric distortions, and blurring, can compromise the clinical and scientific utility of MR images. In this review, we describe sources of off-resonance in MRI, how off-resonance affects images, and strategies to prevent and correct for off-resonance. Given recent advances and the great potential of low field and/or portable MRI, we also …
Magnetic resonance images (MRIs) are widely used to quantify vestibular schwannoma and the cochlea. Recently, deep learning methods have shown state-of-the-art performance for segmenting these structures. However, training segmentation models may require manual labels in target domain, which is expensive and time-consuming. To overcome this problem, domain adaptation is an effective way to leverage information from source domain to obtain accurate segmentations without requiring manual labels in target domain. In this paper, we propose an unsupervised learning framework to segment the VS and c…
The University of Hong Kong（香港大学）；Imperial College London（帝国理工学院）
Abdominal magnetic resonance imaging (MRI) provides a straightforward way of characterizing tissue and locating lesions of patients as in standard diagnosis. However, abdominal MRI often suffers from respiratory motion artifacts, which leads to blurring and ghosting that significantly deteriorate the imaging quality. Conventional methods to reduce or eliminate these motion artifacts include breath holding, patient sedation, respiratory gating, and image post-processing, but these strategies inevitably involve extra scanning time and patient discomfort. In this paper, we propose a novel deep-le…
University of Electronic Science and Technology of China（电子科技大学）；University of Toronto（多伦多大学）；The Hospital for Sick Children (SickKids)（SickKids 儿童医院（病童医院））；Sinai Health System（西奈卫生系统）；University Health Network（大学健康网络）；Lunenfeld-Tanenbaum Research Institute（卢嫩菲尔德-塔南鲍姆研究所）；Sunnybrook Research Institute（桑尼布鲁克研究所）
Data augmentation refers to a group of techniques whose goal is to battle limited amount of available data to improve model generalization and push sample distribution toward the true distribution. While different augmentation strategies and their combinations have been investigated for various computer vision tasks in the context of deep learning, a specific work in the domain of medical imaging is rare and to the best of our knowledge, there has been no dedicated work on exploring the effects of various augmentation methods on the performance of deep learning models in prostate cancer detect…
Brain age estimation based on magnetic resonance imaging (MRI) is an active research area in early diagnosis of some neurodegenerative diseases (e.g. Alzheimer, Parkinson, Huntington, etc.) for elderly people or brain underdevelopment for the young group. Deep learning methods have achieved the state-of-the-art performance in many medical image analysis tasks, including brain age estimation. However, the performance and generalisability of the deep learning model are highly dependent on the quantity and quality of the training data set. Both collecting and annotating brain MRI data are extreme…
Head motion during functional Magnetic Resonance Imaging acquisition can significantly contaminate the neural signal and introduce spurious, distance-dependent changes in signal correlations. This can heavily confound studies of development, aging, and disease. Previous approaches to suppress head motion artifacts have involved sequential regression of nuisance covariates, but this has been shown to reintroduce artifacts. We propose a new motion correction pipeline using an omnibus regression model that avoids this problem by simultaneously regressing out multiple artifacts using the best perf…
Layer Selection in Feature-Based Losses Affects Image Quality and Microstructural Consistency in Deep Learning Super-Resolution of Brain Diffusion MRI
Institue for Applied Medical Informatics, University Medical Center Hamburg-Eppendorf（应用医学信息学研究所，汉堡大学医学中心）；Institute of Computational Neuroscience, University Medical Center Hamburg-Eppendorf（计算神经科学研究所，汉堡大学医学中心）；Center for Biomedical Artificial Intelligence (bAIome), University Medical Center Hamburg-Eppendorf（生物医学人工智能中心（bAIome），汉堡大学医学中心）
Gordon Center for Medical Imaging, Massachusetts General Hospital and Harvard Medical School（马萨诸塞总医院与哈佛医学院戈登医学影像中心）；Texas A&M University（德克萨斯A&M大学）；Texas A&M Institute of Data Science, Texas A&M University（德克萨斯A&M大学数据科学研究所）；Texas A&M University at Qatar（德克萨斯A&M大学卡塔尔分校）
Magnetic resonance imaging (MRI) is one of the noninvasive imaging modalities that can produce high-quality images. However, the scan procedure is relatively slow, which causes patient discomfort and motion artifacts in images. Accelerating MRI hardware is constrained by physical and physiological limitations. A popular alternative approach to accelerated MRI is to undersample the k-space data. While undersampling speeds up the scan procedure, it generates artifacts in the images, and advanced reconstruction algorithms are needed to produce artifact-free images. Recently deep learning has emer…
