We propose a new plug-and-play priors (PnP) based MR image reconstruction method that systematically enforces data consistency while also exploiting deep-learning priors. Our prior is specified through a convolutional neural network (CNN) trained without any artifact-free ground truth to remove undersampling artifacts from MR images. The results on reconstructing free-breathing MRI data into ten respiratory phases show that the method can form high-quality 4D images from severely undersampled measurements corresponding to acquisitions of about 1 and 2 minutes in length. The results also highlight the competitive performance of the method compared to several popular alternatives, including the TGV regularization and traditional UNet3D.
University of Bourgogne Franche-Comte(勃艮第-弗朗什-孔泰大学);CASIS inc.(CASIS公司);University of Sherbrooke(舍布鲁克大学);University Hospital of Dijon(第戎大学医院)
Deep learning methods are the de-facto solutions to a multitude of medical image analysis tasks. Cardiac MRI segmentation is one such application which, like many others, requires a large number of annotated data so a trained network can generalize well. Unfortunately, the process of having a large number of manually curated images by medical experts is both slow and utterly expensive. In this paper, we set out to explore whether expert knowledge is a strict requirement for the creation of annotated datasets that machine learning can successfully train on. To do so, we gauged the performance o…
Washington University in St. Louis(圣路易斯华盛顿大学);Mallinckrodt Institute of Radiology(马林克罗特放射学研究所)
One of the key limitations in conventional deep learning based image reconstruction is the need for registered pairs of training images containing a set of high-quality groundtruth images. This paper addresses this limitation by proposing a novel unsupervised deep registration-augmented reconstruction method (U-Dream) for training deep neural nets to reconstruct high-quality images by directly mapping pairs of unregistered and artifact-corrupted images. The ability of U-Dream to circumvent the need for accurately registered data makes it widely applicable to many biomedical image reconstructio…
We develop and evaluate a neural network-based method for Gibbs artifact and noise removal. A convolutional neural network (CNN) was designed for artifact removal in diffusion-weighted imaging data. Two implementations were considered: one for magnitude images and one for complex images. Both models were based on the same encoder-decoder structure and were trained by simulating MRI acquisitions on synthetic non-MRI images. Both machine learning methods were able to mitigate artifacts in diffusion-weighted images and diffusion parameter maps. The CNN for complex images was also able to reduce a…
Precise segmentation of the liver is critical for computer-aided diagnosis such as pre-evaluation of the liver for living donor-based transplantation surgery. This task is challenging due to the weak boundaries of organs, countless anatomical variations, and the complexity of the background. Computed tomography (CT) scanning and magnetic resonance imaging (MRI) images have different parameters and settings. Thus, images acquired from different modalities differ from one another making liver segmentation challenging task. We propose an efficient liver segmentation with the combination of holist…
Spiral acquisitions are preferred in real-time MRI because of their time efficiency. A fundamental limitation of spirals is image blurring due to off-resonance, which degrades image quality significantly at air-tissue boundaries. Here, we demonstrate a simple CNN-based deblurring method for spiral real-time MRI of human speech production. We show the CNN-based deblurring is capable of restoring blurred vocal tract tissue boundaries, without a need for exam-specific field maps. Deblurring performance is superior to a current auto-calibrated method, and slightly inferior to ideal reconstruction …
Cross-modality image estimation involves the generation of images of one medical imaging modality from that of another modality. Convolutional neural networks (CNNs) have been shown to be useful in identifying, characterising and extracting image patterns. Generative adversarial networks (GANs) use CNNs as generators and estimated images are discriminated as true or false based on an additional network. CNNs and GANs within the image estimation framework may be considered more generally as deep learning approaches, since imaging data tends to be large, leading to a larger number of network wei…
University of Tennessee Health Science Center(田纳西大学健康科学中心);Le Bonheur Children’s Hospital(勒邦赫尔儿童医院);University of Memphis(孟菲斯大学)
Human brain development is rapid during infancy and early childhood. Many disease processes impair this development. Therefore, brain developmental age estimation (BDAE) is essential for all diseases affecting cognitive development. Brain magnetic resonance imaging (MRI) of infants shows brain growth and morphologic patterns during childhood. Therefore, we can estimate the developmental age from brain images. However, MRI analysis is time-consuming because each scan contains millions of data points (voxels). We investigated the three-dimensional convolutional neural network (3D CNN), a deep le…
Early detection of brain metastases (BM) is one of the determining factors for the successful treatment of patients with cancer; however, the accurate detection of small BM lesions (< 15mm) remains a challenging task. We previously described a framework for the detection of small BM in single-sequence gadolinium-enhanced T1-weighted 3D MRI datasets. It combined classical image processing (IP) with a dedicated convolutional neural network, taking approximately 30 seconds to process each dataset due to computation-intensive IP stages. To overcome the speed limitation, this study aims to reformul…
Precise segmentation of knee tissues from magnetic resonance imaging (MRI) is critical in quantitative imaging and diagnosis. Convolutional neural networks (CNNs), which are state of the art, have limitations owing to the lack of image-specific adaptation, such as low tissue contrasts and structural inhomogeneities, thereby leading to incomplete segmentation results. This paper presents a deep learning based automatic segmentation framework for knee tissue segmentation. A novel multipath CNN-based method is proposed, which consists of an encoder decoder-based segmentation network in combinatio…
Philips Research(飞利浦研究院);Philips Healthcare(飞利浦医疗);Leiden University Medical Center(莱顿大学医学中心)
Adaptive intelligence aims at empowering machine learning techniques with the additional use of domain knowledge. In this work, we present the application of adaptive intelligence to accelerate MR acquisition. Starting from undersampled k-space data, an iterative learning-based reconstruction scheme inspired by compressed sensing theory is used to reconstruct the images. We adopt deep neural networks to refine and correct prior reconstruction assumptions given the training data. The network was trained and tested on a knee MRI dataset from the 2019 fastMRI challenge organized by Facebook AI Re…
Despite the recent advances of deep learning algorithms in medical imaging, the automatic segmentation algorithms for kidneys in MRI exams are still scarce. Automated segmentation of kidneys in Magnetic Resonance Imaging (MRI) exams are important for enabling radiomics and machine learning analysis of renal disease. In this work, we propose to use the popular Mask R-CNN for the automatic segmentation of kidneys in coronal T2-weighted Fast Spin Eco slices of 100 MRI exams. We propose the morphological operations as post-processing to further improve the performance of Mask R-CNN for this task. …
Harvard Medical School(哈佛医学院);The University of Sydney(悉尼大学);The University of New South Wales(新南威尔士大学)
The structure and variability of the brain's connections can be investigated via prediction of non-imaging phenotypes using neural networks. However, known neuroanatomical relationships between input features are generally ignored in network design. We propose TractGraphCNN, a novel, anatomically informed graph CNN framework for machine learning tasks using diffusion MRI tractography. An EdgeConv module aggregates features from anatomically similar white matter connections indicated by graph edges, and an attention module enables interpretation of predictive white matter tracts. Results in a s…
Heart Institute (InCor/HC.FMUSP)(心脏研究所(InCor/HC.FMUSP))
CNN (Convolutional Neural Network) models have been successfully used for segmentation of the left ventricle (LV) in cardiac MRI (Magnetic Resonance Imaging), providing clinical measurements. In practice, two questions arise with deployment of CNNs: 1) when is it better to use a shallow model instead of a deeper one? 2) how the size of a dataset might change the network performance? We propose a framework to answer them, by experimenting with deep and shallow versions of three U-Net families, trained from scratch in six subsets varying from 100 to 10,000 images, different network sizes, learni…
Purpose: We aimed to develop deep machine learning (DL) models to improve the detection and segmentation of intraprostatic lesions (IL) on bp-MRI by using whole amount prostatectomy specimen-based delineations. We also aimed to investigate whether transfer learning and self-training would improve results with small amount labelled data. Methods: 158 patients had suspicious lesions delineated on MRI based on bp-MRI, 64 patients had ILs delineated on MRI based on whole mount prostatectomy specimen sections, 40 patients were unlabelled. A non-local Mask R-CNN was proposed to improve the segmentat…
Automated brain tumor detection in MRI images using CNN and ResNet architectures
Department of Computer Science and Engineering, The National Institute of Engineering, Mysuru, Karnataka, India(计算机科学与工程系,国家工程学院,mysuru,卡纳塔克邦,印度);Department of MCA, The National Institute of Engineering, Mysuru, Karnataka, India(MCA系,国家工程学院,mysuru,卡纳塔克邦,印度);Department of Computer Science and Engineering, Vidyavardhaka College of Engineering, Mysuru, Karnataka, India(计算机科学与工程系,维迪瓦尔达卡学院,mysuru,卡纳塔克邦,印度);Department of ECE, Vidyavardhaka College of Engineering, Mysuru, Karnataka, India(电子工程系,维迪瓦尔达卡学院,mysuru,卡纳塔克邦,印度)
University of Kansas(堪萨斯大学);Rochester Institute of Technology(罗切斯特理工学院)
Patient-specific left ventricle (LV) myocardial models have the potential to be used in a variety of clinical scenarios for improved diagnosis and treatment plans. Cine cardiac magnetic resonance (MR) imaging provides high resolution images to reconstruct patient-specific geometric models of the LV myocardium. With the advent of deep learning, accurate segmentation of cardiac chambers from cine cardiac MR images and unsupervised learning for image registration for cardiac motion estimation on a large number of image datasets is attainable. Here, we propose a deep leaning-based framework for th…
University of California, Los Angeles(加利福尼亚大学洛杉矶分校);Cedars-Sinai Medical Center(西达赛奈医疗中心)
High-resolution (HR) magnetic resonance imaging (MRI) provides detailed anatomical information that is critical for diagnosis in the clinical application. However, HR MRI typically comes at the cost of long scan time, small spatial coverage, and low signal-to-noise ratio (SNR). Recent studies showed that with a deep convolutional neural network (CNN), HR generic images could be recovered from low-resolution (LR) inputs via single image super-resolution (SISR) approaches. Additionally, previous works have shown that a deep 3D CNN can generate high-quality SR MRIs by using learned image priors. …
Convolutional neural networks (CNNs) have achieved remarkable success in automatically segmenting organs or lesions on 3D medical images. Recently, vision transformer networks have exhibited exceptional performance in 2D image classification tasks. Compared with CNNs, transformer networks have an appealing advantage of extracting long-range features due to their self-attention algorithm. Therefore, we propose a CNN-Transformer combined model, called BiTr-Unet, with specific modifications for brain tumor segmentation on multi-modal MRI scans. Our BiTr-Unet achieves good performance on the BraTS…
Skolkovo Institute of Science and Technology(斯科尔科沃科学技术研究院)
Deep learning shows high potential for many medical image analysis tasks. Neural networks can work with full-size data without extensive preprocessing and feature generation and, thus, information loss. Recent work has shown that the morphological difference in specific brain regions can be found on MRI with the means of Convolution Neural Networks (CNN). However, interpretation of the existing models is based on a region of interest and can not be extended to voxel-wise image interpretation on a whole image. In the current work, we consider the classification task on a large-scale open-source…
Revisiting Global Token Mixing in Task-Dependent MRI Restoration: Insights from Minimal Gated CNN Baselines
Dept. of Radiology & Imaging Sciences, University of Utah, Salt Lake City, UT, USA(放射学与成像科学系,犹他大学,盐湖城,UT,USA);Dept. of Electrical & Computer Engineering, University of Utah, Salt Lake City, UT, USA(电气与计算机工程系,犹他大学,盐湖城,UT,USA);Dept. of Computer Vision, Mohamed bin Zayed University of Artificial Intelligence, Abu Dhabi, UAE(计算机视觉系,莫扎德人工智能大学,阿布扎赫,EUA);Dept. of Biomedical Engineering, University of Utah, Salt Lake City, UT, USA(生物医学工程系,犹他大学,盐湖城,UT,USA);Dept. of Electrical & Computer Engineering, University of Washington, Seattle, WA, USA(电气与计算机工程系,华盛顿大学,西雅图,WA,USA)
With respect to spatial overlap, CNN-based segmentation of short axis cardiovascular magnetic resonance (CMR) images has achieved a level of performance consistent with inter observer variation. However, conventional training procedures frequently depend on pixel-wise loss functions, limiting optimisation with respect to extended or global features. As a result, inferred segmentations can lack spatial coherence, including spurious connected components or holes. Such results are implausible, violating the anticipated topology of image segments, which is frequently known a priori. Addressing thi…
Schulich School of Engineering, University of Calgary(卡尔加里大学舒立克工程学院);Department of Radiology & Diagnostic Imaging, University of Alberta(阿尔伯塔大学放射与诊断成像系);Alberta Machine Intelligence Institute (AMII), University of Alberta(阿尔伯塔大学阿尔伯塔机器智能研究所(AMII));Mechanical and Manufacturing Engineering, University of Calgary(卡尔加里大学机械与制造工程系)
Objective assessment of Magnetic Resonance Imaging (MRI) scans of osteoarthritis (OA) can address the limitation of the current OA assessment. Segmentation of bone, cartilage, and joint fluid is necessary for the OA objective assessment. Most of the proposed segmentation methods are not performing instance segmentation and suffer from class imbalance problems. This study deployed Mask R-CNN instance segmentation and improved it (improved-Mask R-CNN (iMaskRCNN)) to obtain a more accurate generalized segmentation for OA-associated tissues. Training and validation of the method were performed usi…
Classification of malignancy for breast cancer and other cancer types is usually tackled as an object detection problem: Individual lesions are first localized and then classified with respect to malignancy. However, the drawback of this approach is that abstract features incorporating several lesions and areas that are not labelled as a lesion but contain global medically relevant information are thus disregarded: especially for dynamic contrast-enhanced breast MRI, criteria such as background parenchymal enhancement and location within the breast are important for diagnosis and cannot be cap…
Institute of Mental Health, Singapore(新加坡精神卫生研究院);National University of Singapore(新加坡国立大学);Nanyang Technological University(南洋理工大学);Duke-NUS Medical School(杜克-新加坡国立大学医学院)
Convolutional Neural Network (CNN) has been successfully applied on classification of both natural images and medical images but not yet been applied to differentiating patients with schizophrenia from healthy controls. Given the subtle, mixed, and sparsely distributed brain atrophy patterns of schizophrenia, the capability of automatic feature learning makes CNN a powerful tool for classifying schizophrenia from controls as it removes the subjectivity in selecting relevant spatial features. To examine the feasibility of applying CNN to classification of schizophrenia and controls based on str…
Semantic segmentation using convolutional neural networks (CNNs) is the state-of-the-art for many medical image segmentation tasks including myocardial segmentation in cardiac MR images. However, the predicted segmentation maps obtained from such standard CNN do not allow direct quantification of regional shape properties such as regional wall thickness. Furthermore, the CNNs lack explicit shape constraints, occasionally resulting in unrealistic segmentations. In this paper, we use a CNN to predict shape parameters of an underlying statistical shape model of the myocardium learned from a train…
Magnetic Resonance Imaging (MRI) is one of the most flexible and powerful medical imaging modalities. This flexibility does however come at a cost; MRI images acquired at different sites and with different parameters exhibit significant differences in contrast and tissue appearance, resulting in downstream issues when quantifying brain anatomy or the presence of pathology. In this work, we propose to combine multiparametric MRI-based static-equation sequence simulations with segmentation convolutional neural networks (CNN), to make these networks robust to variations in acquisition parameters.…
Image Sciences Institute, University Medical Center Utrecht, Utrecht University(乌得勒支大学乌得勒支大学医学中心影像科学研究所);Netherlands Cancer Institute – Antoni van Leeuwenhoek Hospital(荷兰癌症研究所——安东尼·范·列文虎克医院)
Breast density, which is the ratio between fibroglandular tissue (FGT) and total breast volume, can be assessed qualitatively by radiologists and quantitatively by computer algorithms. These algorithms often rely on segmentation of breast and FGT volume. In this study, we propose a method to directly assess breast density on MRI, and provide interpretations of these assessments. We assessed breast density in 506 patients with breast cancer using a regression convolutional neural network (CNN). The input for the CNN were slices of breast MRI of 128 x 128 voxels, and the output was a continuous …
High-fidelity semantic segmentation of magnetic resonance volumes is critical for estimating tissue morphometry and relaxation parameters in both clinical and research applications. While manual segmentation is accepted as the gold-standard, recent advances in deep learning and convolutional neural networks (CNNs) have shown promise for efficient automatic segmentation of soft tissues. However, due to the stochastic nature of deep learning and the multitude of hyperparameters in training networks, predicting network behavior is challenging. In this paper, we quantify the impact of three factor…
University of Toronto(多伦多大学);The Hospital for Sick Children(病童医院)
Many high-performance classification models utilize complex CNN-based architectures for Alzheimer's Disease classification. We aim to investigate two relevant questions regarding classification of Alzheimer's Disease using MRI: "Do Vision Transformer-based models perform better than CNN-based models?" and "Is it possible to use a shallow 3D CNN-based model to obtain satisfying results?" To achieve these goals, we propose two models that can take in and process 3D MRI scans: Convolutional Voxel Vision Transformer (CVVT) architecture, and ConvNet3D-4, a shallow 4-block 3D CNN-based model. Our re…
Deep Convolutional Neural Networks (CNNs) are becoming prominent models for semi-automated diagnosis of Alzheimer's Disease (AD) using brain Magnetic Resonance Imaging (MRI). Although being highly accurate, deep CNN models lack transparency and interpretability, precluding adequate clinical reasoning and not complying with most current regulatory demands. One popular choice for explaining deep image models is occluding regions of the image to isolate their influence on the prediction. However, existing methods for occluding patches of brain scans generate images outside the distribution to whi…
National Heart and Lung Institute, Imperial College London(帝国理工学院国家心肺研究所);Imperial College London(帝国理工学院);Sun Yat-sen University(中山大学);Beihang University(北京航空航天大学);Brunel University London(伦敦布鲁内尔大学);University of Cambridge(剑桥大学);Technical University of Munich(慕尼黑工业大学);Weizmann Institute of Science(魏茨曼科学研究所)
Research studies have shown no qualms about using data driven deep learning models for downstream tasks in medical image analysis, e.g., anatomy segmentation and lesion detection, disease diagnosis and prognosis, and treatment planning. However, deep learning models are not the sovereign remedy for medical image analysis when the upstream imaging is not being conducted properly (with artefacts). This has been manifested in MRI studies, where the scanning is typically slow, prone to motion artefacts, with a relatively low signal to noise ratio, and poor spatial and/or temporal resolution. Recen…
3D Brain MRI Classification for Alzheimer Diagnosis Using CNN with Data Augmentation
Vo Thien Nhan Institute of Engineering, Ho Chi Minh City University of Technology (HUTECH), Vietnam(越南胡志明市技术大学(HUTECH)工程学院);Ho Bac Nam Faculty of Computer Science and Engineering, Ho Chi Minh City University of Technology (HCMUT), Vietnam National University Ho Chi Minh City (VNU-HCM), Vietnam(越南胡志明市技术大学(HCMUT)计算机科学与工程学院,越南胡志明市国家大学(VNU-HCM))
Pakistan Institute of Engineering and Applied Sciences (PIEAS)(巴基斯坦工程与应用科学学院);Ibadat International University(伊巴达特国际大学);University of Engineering and Applied Science (UEAS)(工程与应用科学大学)
Brain tumor classification is crucial for clinical analysis and an effective treatment plan to cure patients. Deep learning models help radiologists to accurately and efficiently analyze tumors without manual intervention. However, brain tumor analysis is challenging because of its complex structure, texture, size, location, and appearance. Therefore, a novel deep residual and regional-based Res-BRNet Convolutional Neural Network (CNN) is developed for effective brain tumor (Magnetic Resonance Imaging) MRI classification. The developed Res-BRNet employed Regional and boundary-based operations …
University College London(伦敦大学学院);Massachusetts General Hospital(麻省总医院);Harvard Medical School(哈佛医学院);Copenhagen University Hospital(哥本哈根大学医院);Technical University of Denmark(丹麦技术大学);Massachusetts Institute of Technology(麻省理工学院)
Despite advances in data augmentation and transfer learning, convolutional neural networks (CNNs) difficultly generalise to unseen domains. When segmenting brain scans, CNNs are highly sensitive to changes in resolution and contrast: even within the same MRI modality, performance can decrease across datasets. Here we introduce SynthSeg, the first segmentation CNN robust against changes in contrast and resolution. SynthSeg is trained with synthetic data sampled from a generative model conditioned on segmentations. Crucially, we adopt a domain randomisation strategy where we fully randomise the …
University of California, San Francisco(加州大学旧金山分校)
Purpose: To evaluate the diagnostic utility of two convolutional neural networks (CNNs) for severity staging of anterior cruciate ligament (ACL) injuries. Materials and Methods: This retrospective analysis was conducted on 1243 knee MR images (1008 intact, 18 partially torn, 77 fully torn, and 140 reconstructed ACLs) from 224 patients (age 47 +/- 14 years, 54% women) acquired between 2011 and 2014. The radiologists used a modified scoring metric. To classify ACL injuries with deep learning, two types of CNNs were used, one with three-dimensional (3D) and the other with two-dimensional (2D) con…
University of Electronic Science and Technology of China(电子科技大学);King’s College London(伦敦国王学院);University College London(伦敦大学学院);National Hospital for Neurology and Neurosurgery(国家神经病学和神经外科医院);NVIDIA(英伟达);Queen Square Radiosurgery Centre (Gamma Knife)(女王广场放射外科中心(伽马刀));Neuroimaging Analysis Centre, Queen Square(女王广场神经影像分析中心)
Automatic segmentation of vestibular schwannoma (VS) tumors from magnetic resonance imaging (MRI) would facilitate efficient and accurate volume measurement to guide patient management and improve clinical workflow. The accuracy and robustness is challenged by low contrast, small target region and low through-plane resolution. We introduce a 2.5D convolutional neural network (CNN) able to exploit the different in-plane and through-plane resolutions encountered in standard of care imaging protocols. We use an attention module to enable the CNN to focus on the small target and propose a supervis…
Washington University in St. Louis(圣路易斯华盛顿大学);Mallinckrodt Institute of Radiology(马林克罗特放射学研究所)
Deep equilibrium models (DEQ) have emerged as a powerful alternative to deep unfolding (DU) for image reconstruction. DEQ models-implicit neural networks with effectively infinite number of layers-were shown to achieve state-of-the-art image reconstruction without the memory complexity associated with DU. While the performance of DEQ has been widely investigated, the existing work has primarily focused on the settings where groundtruth data is available for training. We present self-supervised deep equilibrium model (SelfDEQ) as the first self-supervised reconstruction framework for training m…
University of LAquila(拉奎拉大学);Sapienza University of Rome(罗马第一大学)
To date, several automated strategies for identification/segmentation of Multiple Sclerosis (MS) lesions with the use of Magnetic Resonance Imaging (MRI) have been presented, but they are outperformed by human experts, from whom they act very differently. This is mainly due to: the ambiguity originated by MRI instabilities; peculiar variability of MS; non specificity of MRI regarding MS. Physicians partially manage the uncertainty generated by ambiguity relying on radiological/clinical/anatomical background and experience. To emulate human diagnosis, we present an automated framework for ident…
In this paper we develop a Quality Assessment approach for face recognition based on deep learning. The method consists of a Convolutional Neural Network, FaceQnet, that is used to predict the suitability of a specific input image for face recognition purposes. The training of FaceQnet is done using the VGGFace2 database. We employ the BioLab-ICAO framework for labeling the VGGFace2 images with quality information related to their ICAO compliance level. The groundtruth quality labels are obtained using FaceNet to generate comparison scores. We employ the groundtruth data to fine-tune a ResNet-…
Reutlingen University(罗伊特林根大学);Menoufia University(梅努菲亚大学);Karlsruhe Institute of Technology(卡尔斯鲁厄理工学院)
Automatic segmentation is essential for the brain tumor diagnosis, disease prognosis, and follow-up therapy of patients with gliomas. Still, accurate detection of gliomas and their sub-regions in multimodal MRI is very challenging due to the variety of scanners and imaging protocols. Over the last years, the BraTS Challenge has provided a large number of multi-institutional MRI scans as a benchmark for glioma segmentation algorithms. This paper describes our contribution to the BraTS 2022 Continuous Evaluation challenge. We propose a new ensemble of multiple deep learning frameworks namely, De…
From Explanations to Architecture: Explainability-Driven CNN Refinement for Brain Tumor Classification in MRI
American International Brac University Faculty of Psychology University-Bangladesh(美国国际布拉大学心理学学院(孟加拉国));Shinawatra University Department of Computer Science(辛纳瓦特大学计算机科学系);American International North South University(美国国际北南大学);American International University–Bangladesh Department of Computer Science(美国国际大学–孟加拉国计算机科学系);Multimedia University Department of Computer Science(多媒体大学计算机科学系)