Transfer learning refers to machine learning techniques that focus on acquiring knowledge from related tasks to improve generalization in the tasks of interest. In MRI, transfer learning is important for developing strategies that address the variation in MR images. Additionally, transfer learning is beneficial to re-utilize machine learning models that were trained to solve related tasks to the task of interest. Our goal is to identify research directions, gaps of knowledge, applications, and widely used strategies among the transfer learning approaches applied in MR brain imaging. We performed a systematic literature search for articles that applied transfer learning to MR brain imaging. We screened 433 studies and we categorized and extracted relevant information, including task type, application, and machine learning methods. Furthermore, we closely examined brain MRI-specific transfer learning approaches and other methods that tackled privacy, unseen target domains, and unlabeled data. We found 129 articles that applied transfer learning to brain MRI tasks. The most frequent applications were dementia related classification tasks and brain tumor segmentation. A majority of articles utilized transfer learning on convolutional neural networks (CNNs). Only few approaches were clearly brain MRI specific, considered privacy issues, unseen target domains or unlabeled data. We proposed a new categorization to group specific, widely-used approaches. There is an increasing interest in transfer learning within brain MRI. Public datasets have contributed to the popularity of Alzheimer's diagnostics/prognostics and tumor segmentation. Likewise, the availability of pretrained CNNs has promoted their utilization. Finally, the majority of the surveyed studies did not examine in detail the interpretation of their strategies after applying transfer learning, and did not compare to other approaches.
ShanghaiTech University（上海科技大学）；University of Zürich（苏黎世大学）；ETH Zürich（苏黎世联邦理工学院）
Multi-spectral optoacoustic tomography (MSOT) is an emerging optical imaging method providing multiplex molecular and functional information from the rodent brain. It can be greatly augmented by magnetic resonance imaging (MRI) that offers excellent soft-tissue contrast and high-resolution brain anatomy. Nevertheless, registration of multi-modal images remains challenging, chiefly due to the entirely different image contrast rendered by these modalities. Previously reported registration algorithms mostly relied on manual user-dependent brain segmentation, which compromised data interpretation …
University of Pennsylvania（宾夕法尼亚大学）；Columbia University（哥伦比亚大学）；Ludwig-Maximilian University（路德维希-马克西米利安大学）；University of Birmingham（伯明翰大学）
The imaging community has increasingly adopted machine learning (ML) methods to provide individualized imaging signatures related to disease diagnosis, prognosis, and response to treatment. Clinical neuroscience and cancer imaging have been two areas in which ML has offered particular promise. However, many neurologic and neuropsychiatric diseases, as well as cancer, are often heterogeneous in terms of their clinical manifestations, neuroanatomical patterns or genetic underpinnings. Therefore, in such cases, seeking a single disease signature might be ineffectual in delivering individualized p…
Lehigh University（里海大学）；University of Pennsylvania（宾夕法尼亚大学）
Identification of brain regions related to the specific neurological disorders are of great importance for biomarker and diagnostic studies. In this paper, we propose an interpretable Graph Convolutional Network (GCN) framework for the identification and classification of Alzheimer's disease (AD) using multi-modality brain imaging data. Specifically, we extended the Gradient Class Activation Mapping (Grad-CAM) technique to quantify the most discriminative features identified by GCN from brain connectivity patterns. We then utilized them to find signature regions of interest (ROIs) by detecting…
Chulalongkorn University（朱拉隆功大学）；European University of Bangladesh（孟加拉国欧洲大学）；University of Liberal Arts Bangladesh（孟加拉国文科大学）
The growth of abnormal cells in the brain's tissue causes brain tumors. Brain tumors are considered one of the most dangerous disorders in children and adults. It develops quickly, and the patient's survival prospects are slim if not appropriately treated. Proper treatment planning and precise diagnoses are essential to improving a patient's life expectancy. Brain tumors are mainly diagnosed using magnetic resonance imaging (MRI). As part of a convolution neural network (CNN)-based illustration, an architecture containing five convolution layers, five max-pooling layers, a Flatten layer, and t…
School of Biomedical Engineering（生物医学工程学院）；Universidad Autónoma de Ciudad Juárez（华雷斯自治大学）
The brain is a complex organ controlling cognitive process and physical functions. Tumors in the brain are accelerated cell growths affecting the normal function and processes in the brain. MRI scans provides detailed images of the body being one of the most common tests to diagnose brain tumors. The process of segmentation of brain tumors from magnetic resonance imaging can provide a valuable guide for diagnosis, treatment planning and prediction of results. Here we consider the problem brain tumor segmentation using a Deep learning architecture for use in tumor segmentation. Although the pro…
Studying the Effects of Sex-related Differences on Brain Age Prediction using brain MR Imaging
University of Calgary（卡尔加里大学）；Indian Institute of Technology Kharagpur（印度理工学院卡拉格布尔分校）；Dayalbagh Educational Institute (Deemed University)（达亚尔巴格教育学院（被视为大学））
Accurate detection of brain tumors could save lots of lives and increasing the accuracy of this binary classification even as much as a few percent has high importance. Neural Gas Networks (NGN) is a fast, unsupervised algorithm that could be used in data clustering, image pattern recognition, and image segmentation. In this research, we used the metaheuristic Firefly Algorithm (FA) for image contrast enhancement as pre-processing and NGN weights for feature extraction and segmentation of Magnetic Resonance Imaging (MRI) data on two brain tumor datasets from the Kaggle platform. Also, tumor cl…
Clinical diagnostic and treatment decisions rely upon the integration of patient-specific data with clinical reasoning. Cancer presents a unique context that influence treatment decisions, given its diverse forms of disease evolution. Biomedical imaging allows noninvasive assessment of disease based on visual evaluations leading to better clinical outcome prediction and therapeutic planning. Early methods of brain cancer characterization predominantly relied upon statistical modeling of neuroimaging data. Driven by the breakthroughs in computer vision, deep learning became the de facto standar…
University of Sydney（悉尼大学）；Sydney Neuroimaging Analysis Centre（悉尼神经影像分析中心）
Diffusion Weighted Imaging (DWI) is an advanced imaging technique commonly used in neuroscience and neurological clinical research through a Diffusion Tensor Imaging (DTI) model. Volumetric scalar metrics including fractional anisotropy, mean diffusivity, and axial diffusivity can be derived from the DTI model to summarise water diffusivity and other quantitative microstructural information for clinical studies. However, clinical practice constraints can lead to sub-optimal DWI acquisitions with missing slices (either due to a limited field of view or the acquisition of disrupted slices). To a…
So far, numerous learned models have been pressed to use in microwave imaging problems. These models however, are oblivious to the imaging geometry. It has always been hard to bake the physical setup of the imaging array into the structure of the network, resulting in a data-intensive models that are not practical. This work put forward a graph formulation of the microwave imaging array. The architectures proposed is made cognizant of the physical setup, allowing it to incorporate the symmetries, resulting in a less data requirements. Graph convolution and attention mechanism is deployed to ha…
A Foundation Model for Brain Lesion Segmentation with Mixture of Modality Experts
Imperial College London（帝国理工学院）；UK Dementia Research Institute（英国痴呆症研究所）；Beijing Institute of Technology（北京理工大学）；Capital Medical University（首都医科大学）；Beijing Tiantan Hospital（北京天坛医院）
Symmetry Awareness Encoded Deep Learning Framework for Brain Imaging Analysis
School of Computer Science, University of Sydney（悉尼大学计算机科学学院）；Brain and Mind Centre, University of Sydney（悉尼大学脑与心智中心）；Sydney Neuroimaging Analysis Centre（悉尼神经影像分析中心）；School of Mathematics and Statistics, University of Sydney（悉尼大学数学与统计学院）；I-MED Radiology Network（I-MED放射学网络）
Silesian University in Opava（奥帕瓦西里西亚大学）；University of Oxford（牛津大学）；Ostrava University（俄斯特拉发大学）
Matching MRI brain images between patients or mapping patients' MRI slices to the simulated atlas of a brain is key to the automatic registration of MRI of a brain. The ability to match MRI images would also enable such applications as indexing and searching MRI images among multiple patients or selecting images from the region of interest. In this work, we have introduced robustness, accuracy and cumulative distance metrics and methodology that allows us to compare different techniques and approaches in matching brain MRI of different patients or matching MRI brain slice to a position in the …
Research & Development Institute of Northwestern Polytechnical University in Shenzhen（西北工业大学深圳研究院）；Northwestern Polytechnical University（西北工业大学）；University of Pittsburgh（匹兹堡大学）；University of Sydney（悉尼大学）；JD Finance America Corporation（京东金融美国公司）
In our previous work, $i.e.$, HNF-Net, high-resolution feature representation and light-weight non-local self-attention mechanism are exploited for brain tumor segmentation using multi-modal MR imaging. In this paper, we extend our HNF-Net to HNF-Netv2 by adding inter-scale and intra-scale semantic discrimination enhancing blocks to further exploit global semantic discrimination for the obtained high-resolution features. We trained and evaluated our HNF-Netv2 on the multi-modal Brain Tumor Segmentation Challenge (BraTS) 2021 dataset. The result on the test set shows that our HNF-Netv2 achieved…
National Yang Ming Chiao Tung University（国立阳明交通大学）
Electrical impedance tomography (EIT) is a promising technique for biomedical imaging. The strength of EIT is its ability to reconstruct images of the body's internal structures through radiation-safe techniques. EIT is regarded as safe for patients' health, and it is currently being actively researched. This paper investigates the application of EIT during deep brain stimulation (DBS) surgery as a means to identify targets during operations. DBS involves a surgical procedure in which a lead or electrode array is implanted in a specific target area in the brain. Electrical stimulations are the…
Research & Development Institute of Northwestern Polytechnical University in Shenzhen（西北工业大学深圳研究院）；Northwestern Polytechnical University（西北工业大学）；University of Pittsburgh（匹兹堡大学）；University of Sydney（悉尼大学）；JD Finance America Corporation（京东金融美国公司）
In this paper, we propose a Hybrid High-resolution and Non-local Feature Network (H2NF-Net) to segment brain tumor in multimodal MR images. Our H2NF-Net uses the single and cascaded HNF-Nets to segment different brain tumor sub-regions and combines the predictions together as the final segmentation. We trained and evaluated our model on the Multimodal Brain Tumor Segmentation Challenge (BraTS) 2020 dataset. The results on the test set show that the combination of the single and cascaded models achieved average Dice scores of 0.78751, 0.91290, and 0.85461, as well as Hausdorff distances ($95\%$…
The imaging and subsequent accurate diagnosis of paediatric brain tumours presents a radiological challenge, with magnetic resonance imaging playing a key role in providing tumour specific imaging information. Diffusion weighted and perfusion imaging are commonly used to aid the non invasive diagnosis of paediatric brain tumours, but are usually evaluated by expert qualitative review. Quantitative studies are mainly single centre and single modality. The aim of this work was to combine multi centre diffusion and perfusion imaging, with machine learning, to develop machine learning based classi…
A large-scale heterogeneous 3D magnetic resonance brain imaging dataset for self-supervised learning
Department of Computer Science, University of Copenhagen（计算机科学系，哥本哈根大学）；Pioneer Centre For AI（人工智能先锋中心）；Copenhagen Research Centre for Biological and Precision Psychiatry（哥本哈根生物与精准精神病学研究中心）；Mental Health Centre Copenhagen（哥本哈根心理健康中心）；Copenhagen University Hospital（哥本哈根大学医院）；Rigshospitalet（皇家医院）；Radiological AI Testcenter（放射学AI测试中心）；Athinoula A. Martinos Center for Biomedical Imaging（Athinoula A. Martinos生物医学成像中心）；Massachusetts General Hospital and Harvard Medical School（麻省总医院和哈佛医学院）；Johns Hopkins University（约翰霍普金斯大学）
AI generated annotations for Breast, Brain, Liver, Lungs and Prostate cancer collections in National Cancer Institute Imaging Data Commons
BAMF Health（BAMF健康公司）；National Institute of Health（美国国立卫生研究院）；Frederick National Laboratory for Cancer Research（弗雷德里克国家癌症研究实验室）；Brigham and Women’s Hospital（布莱根妇女医院）；Harvard Medical School（哈佛医学院）
National Institute of Mental Health（美国国家精神卫生研究所）；Birkbeck College（伯克贝克学院）；University College London（伦敦大学学院）
Deep neural networks have brought remarkable breakthroughs in medical image analysis. However, due to their data-hungry nature, the modest dataset sizes in medical imaging projects might be hindering their full potential. Generating synthetic data provides a promising alternative, allowing to complement training datasets and conducting medical image research at a larger scale. Diffusion models recently have caught the attention of the computer vision community by producing photorealistic synthetic images. In this study, we explore using Latent Diffusion Models to generate synthetic images from…
Medical image segmentation has become an essential technique in clinical and research-oriented applications. Because manual segmentation methods are tedious, and fully automatic segmentation lacks the flexibility of human intervention or correction, semi-automatic methods have become the preferred type of medical image segmentation. We present a hybrid, semi-automatic segmentation method in 3D that integrates both region-based and boundary-based procedures. Our method differs from previous hybrid methods in that we perform region-based and boundary-based approaches separately, which allows for…
Massachusetts General Hospital（麻省总医院）；Harvard Medical School（哈佛医学院）；Yale University（耶鲁大学）；University of Iowa（爱荷华大学）；Augusta University（奥古斯塔大学）；University of Texas Health Science Center at Houston（德克萨斯大学休斯顿健康科学中心）；Johns Hopkins University（约翰斯·霍普金斯大学）
Rodent stroke models are important for evaluating treatments and understanding the pathophysiology and behavioral changes of brain ischemia, and magnetic resonance imaging (MRI) is a valuable tool for measuring outcome in preclinical studies. Brain extraction is an essential first step in most neuroimaging pipelines; however, it can be challenging in the presence of severe pathology and when dataset quality is highly variable. Convolutional neural networks (CNNs) can improve accuracy and reduce operator time, facilitating high throughput preclinical studies. As part of an ongoing preclinical s…
This article is based on the first chapter of book Chung (2013), where brain and medical images are introduced. The most widely used brain imaging modalities are magnetic resonance images (MRI), functional-MRI (fMRI) and diffusion tensor images (DTI). A brief introduction to each imaging modality is explained. Further, we explain what kind of curve, volume and surface data that can be extracted from each modality.
Modern neuroimaging techniques provide us with unique views on brain structure and function; i.e., how the brain is wired, and where and when activity takes place. Data acquired using these techniques can be analyzed in terms of its network structure to reveal organizing principles at the systems level. Graph representations are versatile models where nodes are associated to brain regions and edges to structural or functional connections. Structural graphs model neural pathways in white matter that are the anatomical backbone between regions. Functional graphs are built based on functional con…
Spectral imaging paves way for various fields and particular in biomedical research. However, spectral imaging mainly depending on spatial or temporal scanning, cannot achieve high temporal, spatial and spectral resolution simultaneously. In this study, we demonstrated a silicon real-time ultra-spectral imaging chip based on reconfigurable metasurfaces, comprising of 155,216 (356$\times$436) image-adaptive micro-spectrometers with ultra-high center-wavelength accuracy of 0.04 nm and spectral resolution of 0.8 nm. It is employed for imaging brain hemodynamics, and the dynamic spectral absorptio…
Mayo Clinic（梅奥诊所）；Moffitt Cancer Center and Research Institute（莫菲特癌症中心与研究所）
Whole brain extraction, also known as skull stripping, is a process in neuroimaging in which non-brain tissue such as skull, eyeballs, skin, etc. are removed from neuroimages. Skull striping is a preliminary step in presurgical planning, cortical reconstruction, and automatic tumor segmentation. Despite a plethora of skull stripping approaches in the literature, few are sufficiently accurate for processing pathology-presenting MRIs, especially MRIs with brain tumors. In this work we propose a deep learning approach for skull striping common MRI sequences in oncology such as T1-weighted with ga…
PUCRS（南里奥格兰德州天主教大学）；BraIns, Brain Institute of Rio Grande do Sul（南里奥格兰德州大脑研究所）
Brain imaging of mental health, neurodevelopmental and learning disorders has coupled with machine learning to identify patients based only on their brain activation, and ultimately identify features that generalize from smaller samples of data to larger ones. However, the success of machine learning classification algorithms on neurofunctional data has been limited to more homogeneous data sets of dozens of participants. More recently, larger brain imaging data sets have allowed for the application of deep learning techniques to classify brain states and clinical groups solely from neurofunct…
CEST suffers from two main problems long acquisitin times or restricted coverage as well as incoherent protocol settings. In this paper we give suggestions on how to optimise your protocol settings fro CEST and present one setting for APT CEST. To increase the coverage while keeping the acquisition time constant we suggest using a spatial temporal Compressed Sensing approach. Finally, 1.8mm isotropic whole brain APT CEST maps can be acquired in a little bit less than 2min with a fully integrated online reconstruction. This will pave the way to an even further clinical use of CEST.
Diffusion weighted imaging techniques permit us to infer microstructural detail in biological tissue in vivo and noninvasively. Modern sequences are based on advanced diffusion encoding schemes, allowing probing of more revealing measures of tissue microstructure than the standard apparent diffusion coefficient or fractional anisotropy. Though these methods may result in faster or more revealing acquisitions, they generally demand prior knowledge of sequence-specific parameters for which there is no accepted sharing standard. Here, we present a metadata labelling scheme suitable for the needs …
In this paper, we studied the association between the change of structural brain volumes to the potential development of Alzheimer's disease (AD). Using a simple abstraction technique, we converted regional cortical and subcortical volume differences over two time points for each study subject into a graph. We then obtained substructures of interest using a graph decomposition algorithm in order to extract pivotal nodes via multi-view feature selection. Intensive experiments using robust classification frameworks were conducted to evaluate the performance of using the brain substructures obtai…
Magnetic Resonance Imaging (MRI) is one of the most flexible and powerful medical imaging modalities. This flexibility does however come at a cost; MRI images acquired at different sites and with different parameters exhibit significant differences in contrast and tissue appearance, resulting in downstream issues when quantifying brain anatomy or the presence of pathology. In this work, we propose to combine multiparametric MRI-based static-equation sequence simulations with segmentation convolutional neural networks (CNN), to make these networks robust to variations in acquisition parameters.…
Vanderbilt University（范德堡大学）；Vanderbilt University Institute of Imaging Science（范德堡大学影像科学研究所）
Veterans with mild traumatic brain injury (mTBI) have reported auditory and visual dysfunction that persists beyond the acute incident. The etiology behind these symptoms is difficult to characterize with current clinical imaging. These functional deficits may be caused by shear injury or micro-bleeds, which can be detected with special imaging modalities. We explore these hypotheses in a pilot study of multi-parametric MRI. We extract over 1,000 imaging and clinical metrics and project them to a low-dimensional space, where we can discriminate between healthy controls and patients with mTBI. …
Photoacoustic tomography (PAT) is a newly developed medical imaging modality, which combines the advantages of pure optical imaging and ultrasound imaging, owning both high optical contrast and deep penetration depth. Very recently, PAT is studied in human brain imaging. Nevertheless, while ultrasound waves are passing through the human skull tissues, the strong acoustic attenuation and aberration will happen, which causes photoacoustic signals' distortion. In this work, we use 10 magnetic resonance angiography (MRA) human brain volumes, and manually segment them to obtain the 2D human brain n…
Attenuation and scatter correction (AC) is crucial for quantitative Positron Emission Tomography (PET) imaging. Recently, direct application of AC in the image domain using deep learning approaches has been proposed for the hybrid PET/MR and dedicated PET systems that lack accompanying transmission or anatomical imaging. This study set out to investigate deep learning-based AC in the image domain using different input settings.
During the first years of life, the human brain undergoes dynamic spatially-heterogeneous changes, involving differentiation of neuronal types, dendritic arborization, axonal ingrowth, outgrowth and retraction, synaptogenesis, and myelination. To better quantify these changes, this article presents a method for probing tissue microarchitecture by characterizing water diffusion in a spectrum of length scales, factoring out the effects of intra-voxel orientation heterogeneity. Our method is based on the spherical means of the diffusion signal, computed over gradient directions for a fixed set of…
Brain Imaging Data Structure (BIDS) allows the user to organise brain imaging data into a clear and easy standard directory structure. BIDS is widely supported by the scientific community and is considered a powerful standard for management. The original BIDS is limited to images or data related to the brain. Medical Imaging Data Structure (MIDS) was therefore conceived with the objective of extending this methodology to other anatomical regions and other types of imaging systems in these areas.
