As deep learning is showing unprecedented success in medical image analysis tasks, the lack of sufficient medical data is emerging as a critical problem. While recent attempts to solve the limited data problem using Generative Adversarial Networks (GAN) have been successful in generating realistic images with diversity, most of them are based on image-to-image translation and thus require extensive datasets from different domains. Here, we propose a novel model that can successfully generate 3D brain MRI data from random vectors by learning the data distribution. Our 3D GAN model solves both image blurriness and mode collapse problems by leveraging alpha-GAN that combines the advantages of Variational Auto-Encoder (VAE) and GAN with an additional code discriminator network. We also use the Wasserstein GAN with Gradient Penalty (WGAN-GP) loss to lower the training instability. To demonstrate the effectiveness of our model, we generate new images of normal brain MRI and show that our model outperforms baseline models in both quantitative and qualitative measurements. We also train the model to synthesize brain disorder MRI data to demonstrate the wide applicability of our model. Our results suggest that the proposed model can successfully generate various types and modalities of 3D whole brain volumes from a small set of training data.
Hamburg University of Technology（汉堡工业大学）；Jung Diagnostics GmbH（荣格诊断有限公司）
The use of supervised deep learning techniques to detect pathologies in brain MRI scans can be challenging due to the diversity of brain anatomy and the need for annotated data sets. An alternative approach is to use unsupervised anomaly detection, which only requires sample-level labels of healthy brains to create a reference representation. This reference representation can then be compared to unhealthy brain anatomy in a pixel-wise manner to identify abnormalities. To accomplish this, generative models are needed to create anatomically consistent MRI scans of healthy brains. While recent di…
Segmenting the fine structure of the mouse brain on magnetic resonance (MR) images is critical for delineating morphological regions, analyzing brain function, and understanding their relationships. Compared to a single MRI modality, multimodal MRI data provide complementary tissue features that can be exploited by deep learning models, resulting in better segmentation results. However, multimodal mouse brain MRI data is often lacking, making automatic segmentation of mouse brain fine structure a very challenging task. To address this issue, it is necessary to fuse multimodal MRI data to produ…
Towards General Text-guided Image Synthesis for Customized Multimodal Brain MRI Generation
ShanghaiTech University（上海科技大学）；Shanghai Clinical Research and Trial Center（上海临床研究与试验中心）；Affiliated Hangzhou First People’s Hospital, Xihu University School of Medicine（西湖大学医学院附属杭州市第一人民医院）
Brain age estimation based on magnetic resonance imaging (MRI) is an active research area in early diagnosis of some neurodegenerative diseases (e.g. Alzheimer, Parkinson, Huntington, etc.) for elderly people or brain underdevelopment for the young group. Deep learning methods have achieved the state-of-the-art performance in many medical image analysis tasks, including brain age estimation. However, the performance and generalisability of the deep learning model are highly dependent on the quantity and quality of the training data set. Both collecting and annotating brain MRI data are extreme…
LPIXEL Inc.（LPIXEL公司）；University of Cambridge（剑桥大学）；National Institute of Informatics（国立信息学研究所）；National Center for Global Health and Medicine（国立国际医疗研究中心）；Juntendo University（顺天堂大学）；ELTE Eötvös Loránd University（厄特沃什·罗兰大学）；The University of Tokyo（东京大学）
Unsupervised learning can discover various unseen abnormalities, relying on large-scale unannotated medical images of healthy subjects. Towards this, unsupervised methods reconstruct a 2D/3D single medical image to detect outliers either in the learned feature space or from high reconstruction loss. However, without considering continuity between multiple adjacent slices, they cannot directly discriminate diseases composed of the accumulation of subtle anatomical anomalies, such as Alzheimer's Disease (AD). Moreover, no study has shown how unsupervised anomaly detection is associated with eith…
FLAIRBrainSeg: Fine-grained brain segmentation using FLAIR MRI only
Univ. Bordeaux（波尔多大学）；Bordeaux INP（波尔多国立综合理工学院）；CNRS（法国国家科学研究中心）；Inserm（法国国家健康与医学研究院）；CHU de Bordeaux（波尔多大学医院中心）；Universitat Politècnica de València（瓦伦西亚理工大学）；Instituto de Aplicaciones de las Tecnologías de la Información y de las Comunicaciones Avanzadas（高级信息技术与通信应用研究所）
Institute of Medical Technology and Intelligent Systems, Hamburg University of Technology（汉堡工业大学医疗技术与智能系统研究所）；jung diagnostics GmbH（jung diagnostics GmbH公司）；Hamburg University of Technology（汉堡工业大学）
Lesion detection in brain Magnetic Resonance Images (MRIs) remains a challenging task. MRIs are typically read and interpreted by domain experts, which is a tedious and time-consuming process. Recently, unsupervised anomaly detection (UAD) in brain MRI with deep learning has shown promising results to provide a quick, initial assessment. So far, these methods only rely on the visual appearance of healthy brain anatomy for anomaly detection. Another biomarker for abnormal brain development is the deviation between the brain age and the chronological age, which is unexplored in combination with …
Deep learning builds the foundation for many medical image analysis tasks where neuralnetworks are often designed to have a large receptive field to incorporate long spatialdependencies. Recent work has shown that large receptive fields are not always necessaryfor computer vision tasks on natural images. We explore whether this translates to certainmedical imaging tasks such as age and sex prediction from a T1-weighted brain MRI scans.
Silesian University in Opava（奥帕瓦西里西亚大学）；University of Oxford（牛津大学）；Ostrava University（俄斯特拉发大学）
Matching MRI brain images between patients or mapping patients' MRI slices to the simulated atlas of a brain is key to the automatic registration of MRI of a brain. The ability to match MRI images would also enable such applications as indexing and searching MRI images among multiple patients or selecting images from the region of interest. In this work, we have introduced robustness, accuracy and cumulative distance metrics and methodology that allows us to compare different techniques and approaches in matching brain MRI of different patients or matching MRI brain slice to a position in the …
Image-based brain cancer prediction models, based on radiomics, quantify the radiologic phenotype from magnetic resonance imaging (MRI). However, these features are difficult to reproduce because of variability in acquisition and preprocessing pipelines. Despite evidence of intra-tumor phenotypic heterogeneity, the spatial diversity between different slices within an MRI scan has been relatively unexplored using such methods. In this work, we propose a deep anchor attention aggregation strategy with a Vision Transformer to predict survival risk for brain cancer patients. A Deep Anchor Attentio…
Metadata-Conditioned Generative Models to Synthesize Anatomically-Plausible 3D Brain MRIs
Stanford University（斯坦福大学）；University of Missouri（密苏里大学）；Missouri Institute of Mental Health（密苏里精神卫生研究所）；Weill Cornell Medicine（威尔康奈尔医学院）；SRI International（SRI国际研究所）
Imperial College London（伦敦帝国学院）；King’s College London（伦敦国王学院）；FAU Erlangen-Nürnberg（埃尔朗根-纽伦堡大学）；Technical University of Munich（慕尼黑工业大学）
Fetal Magnetic Resonance Imaging (MRI) is used in prenatal diagnosis and to assess early brain development. Accurate segmentation of the different brain tissues is a vital step in several brain analysis tasks, such as cortical surface reconstruction and tissue thickness measurements. Fetal MRI scans, however, are prone to motion artifacts that can affect the correctness of both manual and automatic segmentation techniques. In this paper, we propose a novel network structure that can simultaneously generate conditional atlases and predict brain tissue segmentation, called CAS-Net. The condition…
The Hong Kong University of Science and Technology（香港科技大学）
Registration of pre-operative and follow-up brain MRI scans is challenging due to the large variation of tissue appearance and missing correspondences in tumour recurrence regions caused by tumour mass effect. Although recent deep learning-based deformable registration methods have achieved remarkable success in various medical applications, most of them are not capable of registering images with pathologies. In this paper, we propose a 3-step registration pipeline for pre-operative and follow-up brain MRI scans that consists of 1) a multi-level affine registration, 2) a conditional deep Lapla…
Institute of Medical Technology and Intelligent Systems, Hamburg University of Technology（汉堡工业大学医学技术与智能系统研究所）；jung diagnostics GmbH（jung diagnostics 有限公司）
Purpose. Brain Magnetic Resonance Images (MRIs) are essential for the diagnosis of neurological diseases. Recently, deep learning methods for unsupervised anomaly detection (UAD) have been proposed for the analysis of brain MRI. These methods rely on healthy brain MRIs and eliminate the requirement of pixel-wise annotated data compared to supervised deep learning. While a wide range of methods for UAD have been proposed, these methods are mostly 2D and only learn from MRI slices, disregarding that brain lesions are inherently 3D and the spatial context of MRI volumes remains unexploited. Metho…
Hosei University（法政大学）；Johns Hopkins University School of Medicine（约翰斯·霍普金斯大学医学院）
Increasing numbers of MRI brain scans, improvements in image resolution, and advancements in MRI acquisition technology are causing significant increases in the demand for and burden on radiologists' efforts in terms of reading and interpreting brain MRIs. Content-based image retrieval (CBIR) is an emerging technology for reducing this burden by supporting the reading of medical images. High dimensionality is a major challenge in developing a CBIR system that is applicable for 3D brain MRIs. In this study, we propose a system called disease-oriented data concentration with metric learning (DDC…
Rewiring Development in Brain Segmentation: Leveraging Adult Brain Priors for Enhancing Infant MRI Segmentation
Department of Information Engineering University of Brescia (Italy)（信息工程系，布雷西亚大学）；School of Psychology & Neuroscience University of Glasgow (UK)（心理学与神经科学学院，格拉斯哥大学）；Department of Medical and Surgical Specialties, Radiological Sciences, and Public Health University of Brescia (Italy)（医学与外科专科、放射科学与公共卫生系，布雷西亚大学）
MIT（麻省理工学院）；Boston Children’s Hospital（波士顿儿童医院）；Harvard Medical School（哈佛医学院）
Fetal brain MRI is useful for diagnosing brain abnormalities but is challenged by fetal motion. The current protocol for T2-weighted fetal brain MRI is not robust to motion so image volumes are degraded by inter- and intra- slice motion artifacts. Besides, manual annotation for fetal MR image quality assessment are usually time-consuming. Therefore, in this work, a semi-supervised deep learning method that detects slices with artifacts during the brain volume scan is proposed. Our method is based on the mean teacher model, where we not only enforce consistency between student and teacher model…
The University of California San Francisco Brain Metastases Stereotactic Radiosurgery (UCSF-BMSR) MRI Dataset
University of California San Francisco（加州大学旧金山分校）；University of California, San Diego（加州大学圣地亚哥分校）；University of Pennsylvania（宾夕法尼亚大学）；Duke University（杜克大学）
Martinos Center for Biomedical Imaging, Massachusetts General Hospital, Harvard Medical School（哈佛医学院麻省总医院Martinos生物医学影像中心）；Massachusetts Institute of Technology（麻省理工学院）；Cornell University（康奈尔大学）；University College London（伦敦大学学院）
Probabilistic atlas priors have been commonly used to derive adaptive and robust brain MRI segmentation algorithms. Widely-used neuroimage analysis pipelines rely heavily on these techniques, which are often computationally expensive. In contrast, there has been a recent surge of approaches that leverage deep learning to implement segmentation tools that are computationally efficient at test time. However, most of these strategies rely on learning from manually annotated images. These supervised deep learning methods are therefore sensitive to the intensity profiles in the training dataset. To…
SynthBA: Reliable Brain Age Estimation Across Multiple MRI Sequences and Resolutions
Dept. of Math and Computer Science（数学与计算机科学系）；Department of Neuroinflammation（神经炎症系）；School of Physics, Engineering and CS（物理、工程与计算机科学学校）；University of Hertfordshire（赫特福德大学）
Indian Institute of Technology, Madras（马德拉斯印度理工学院）；Healthcare Technology Innovation Centre, IIT Madras（印度理工学院马德拉斯分校医疗技术创新中心）
Anomaly detection in MRI is of high clinical value in imaging and diagnosis. Unsupervised methods for anomaly detection provide interesting formulations based on reconstruction or latent embedding, offering a way to observe properties related to factorization. We study four existing modeling methods, and report our empirical observations using simple data science tools, to seek outcomes from the perspective of factorization as it would be most relevant to the task of unsupervised anomaly detection, considering the case of brain structural MRI. Our study indicates that anomaly detection algorit…
Comparative Evaluation of Transfer Learning for Classification of Brain Tumor Using MRI
Florida Institute of Technology（佛罗里达理工学院）；Daffodil International University（达福德国际大学）；University of North Texas（北德克萨斯大学）；The University of Alabama at Birmingham（阿拉巴马大学伯明翰分校）
Evaluation of 3D Counterfactual Brain MRI Generation
Wu Tsai Neurosciences Institute, Stanford University, Stanford, CA, USA Department of Radiology, Stanford University, Stanford, CA, USA Dept. of Psychiatry \& Behavioral Sciences, Stanford University, Stanford, CA, USA Department of Computer Science, Stanford University, Stanford, CA, USA Department of Bioengineering, Stanford University, Stanford, CA, USA Department of Electrical Engineering, Stanford University, Stanford, CA, USA
University of Pennsylvania（宾夕法尼亚大学）；Lund University（隆德大学）
Ex vivo MRI of the brain provides remarkable advantages over in vivo MRI for visualizing and characterizing detailed neuroanatomy. However, automated cortical segmentation methods in ex vivo MRI are not well developed, primarily due to limited availability of labeled datasets, and heterogeneity in scanner hardware and acquisition protocols. In this work, we present a high resolution 7 Tesla dataset of 32 ex vivo human brain specimens. We benchmark the cortical mantle segmentation performance of nine neural network architectures, trained and evaluated using manually-segmented 3D patches sampled…
Objective: Magnetic resonance imaging (MRI) has been widely used for the analysis and diagnosis of brain diseases. Accurate and automatic brain tumor segmentation is of paramount importance for radiation treatment. However, low tissue contrast in tumor regions makes it a challenging task.Approach: We propose a novel segmentation network named Category Guided Attention U-Net (CGA U-Net). In this model, we design a Supervised Attention Module (SAM) based on the attention mechanism, which can capture more accurate and stable long-range dependency in feature maps without introducing much computati…
Vingroup Big Data Institute (VinBigdata)（越南 VinBigdata 大数据研究院）；Hanoi University of Science and Technology (HUST)（河内科技大学）；University of Engineering and Technology (UET), VNU（越南国立大学工程技术大学）
Brain tumor segmentation plays an essential role in medical image analysis. In recent studies, deep convolution neural networks (DCNNs) are extremely powerful to tackle tumor segmentation tasks. We propose in this paper a novel training method that enhances the segmentation results by adding an additional classification branch to the network. The whole network was trained end-to-end on the Multimodal Brain Tumor Segmentation Challenge (BraTS) 2020 training dataset. On the BraTS's validation set, it achieved an average Dice score of 78.43%, 89.99%, and 84.22% respectively for the enhancing tumo…
Hamburg University of Technology（汉堡工业大学）；Jung Diagnostics GmbH（荣格诊断有限公司）
The detection of lesions in magnetic resonance imaging (MRI)-scans of human brains remains challenging, time-consuming and error-prone. Recently, unsupervised anomaly detection (UAD) methods have shown promising results for this task. These methods rely on training data sets that solely contain healthy samples. Compared to supervised approaches, this significantly reduces the need for an extensive amount of labeled training data. However, data labelling remains error-prone. We study how unhealthy samples within the training data affect anomaly detection performance for brain MRI-scans. For our…
InverseSR: 3D Brain MRI Super-Resolution Using a Latent Diffusion Model
St. Francis Xavier University（圣弗朗西斯泽维尔大学）；Martinos Center for Biomedical Imaging（马蒂诺生物医学成像中心）；Massachusetts General Hospital（麻省总医院）；Nova Scotia Health Authority（新斯科舍省卫生局）；King’s College London（伦敦国王学院）；University of California, Santa Cruz（加州大学圣克鲁兹分校）
University College Dublin（都柏林大学学院）；Université Paris-Est Créteil Val de Marne - Université Paris 12（巴黎东-马恩拉瓦莱大学（巴黎第十二大学））；ADAPT SFI Research Centre（ADAPT SFI研究中心）
Magnetic Resonance Imaging (MRI) is the most commonly used non-intrusive technique for medical image acquisition. Brain tumor segmentation is the process of algorithmically identifying tumors in brain MRI scans. While many approaches have been proposed in the literature for brain tumor segmentation, this paper proposes a lightweight implementation of U-Net. Apart from providing real-time segmentation of MRI scans, the proposed architecture does not need large amount of data to train the proposed lightweight U-Net. Moreover, no additional data augmentation step is required. The lightweight U-Ne…
Alzheimer's disease (AD) is one of the most common public health issues the world is facing today. This disease has a high prevalence primarily in the elderly accompanying memory loss and cognitive decline. AD detection is a challenging task which many authors have developed numerous computerized automatic diagnosis systems utilizing neuroimaging and other clinical data. MRI scans provide high-intensity visible features, making these scans the most widely used brain imaging technique. In recent years deep learning has achieved leading success in medical image analysis. But a relatively little …
Columbia University（哥伦比亚大学）；Columbia University Irving Medical Center（哥伦比亚大学欧文医学中心）；Zuckerman Institute（扎克曼研究所）
Cerebral blood volume (CBV) is a hemodynamic correlate of oxygen metabolism and reflects brain activity and function. High-resolution CBV maps can be generated using the steady-state gadolinium-enhanced MRI technique. Such a technique requires an intravenous injection of exogenous gadolinium based contrast agent (GBCA) and recent studies suggest that the GBCA can accumulate in the brain after frequent use. We hypothesize that endogenous sources of contrast might exist within the most conventional and commonly acquired structural MRI, potentially obviating the need for exogenous contrast. Here,…
Improving cross-domain brain tissue segmentation in fetal MRI with synthetic data
Lausanne University Hospital（洛桑大学医院）；University of Lausanne (UNIL)（洛桑大学）；CIBM Center for Biomedical Imaging（CIBM生物医学成像中心）；King’s College London（伦敦国王学院）；FAU Erlangen-Nuremberg（埃尔朗根-纽伦堡大学）
Rensselaer Polytechnic Institute（伦斯勒理工学院）；Wake Forest School of Medicine（维克森林大学医学院）
Treatment decisions for brain metastatic disease rely on knowledge of the primary organ site, and currently made with biopsy and histology. Here we develop a novel deep learning approach for accurate non-invasive digital histology with whole-brain MRI data. Our IRB-approved single-site retrospective study was comprised of patients (n=1,399) referred for MRI treatment-planning and gamma knife radiosurgery over 21 years. Contrast-enhanced T1-weighted and T2-weighted Fluid-Attenuated Inversion Recovery brain MRI exams (n=1,582) were preprocessed and input to the proposed deep learning workflow fo…
Automatic segmentation of brain abnormalities is challenging, as they vary considerably from one pathology to another. Current methods are supervised and require numerous annotated images for each pathology, a strenuous task. To tackle anatomical variability, Unsupervised Anomaly Detection (UAD) methods are proposed, detecting anomalies as outliers of a healthy model learned using a Variational Autoencoder (VAE). Previous work on UAD adopted a 2D approach, meaning that MRIs are processed as a collection of independent slices. Yet, it does not fully exploit the spatial information contained in …
Fordham University（福特汉姆大学）；QS Investors, LLC（QS投资有限责任公司）；Grossman Medical School（格罗斯曼医学院）
In-scanner motion degrades the quality of magnetic resonance imaging (MRI) thereby reducing its utility in the detection of clinically relevant abnormalities. We introduce a deep learning-based MRI artifact reduction model (DMAR) to localize and correct head motion artifacts in brain MRI scans. Our approach integrates the latest advances in object detection and noise reduction in Computer Vision. Specifically, DMAR employs a two-stage approach: in the first, degraded regions are detected using the Single Shot Multibox Detector (SSD), and in the second, the artifacts within the found regions ar…
Escola Politécnica, Universidade de São Paulo（圣保罗大学理工学院）；Brain Institute, IIEP Albert Einstein（阿尔伯特·爱因斯坦教育与研究学院脑研究所）；Faculdade de Medicina, Universidade de São Paulo（圣保罗大学医学院）；University of California San Francisco（加州大学旧金山分校）；University of Würzburg（维尔茨堡大学）；Athinoula A. Martinos Center for Biomedical Imaging, MGH/HST（麻省总医院/哈佛-麻省理工健康科学与技术部阿西诺拉·A·马丁诺斯生物医学成像中心）
Although the latest advances in MRI technology have allowed the acquisition of higher resolution images, reliable delineation of cytoarchitectural or subcortical nuclei boundaries is not possible. As a result, histological images are still required to identify the exact limits of neuroanatomical structures. However, histological processing is associated with tissue distortion and fixation artifacts, which prevent a direct comparison between the two modalities. Our group has previously proposed a histological procedure based on celloidin embedding that reduces the amount of artifacts and yields…
