A Survey on Deep Learning for Neuroimaging-based Brain Disorder Analysis
College of Computer Science and Technology, Nanjing Forestry University（南京林业大学计算机科学与技术学院）；College of Computer Science and Technology, Nanjing University of Aeronautics and Astronautics（南京航空航天大学计算机科学与技术学院）；University of North Carolina at Chapel Hill（北卡罗来纳大学教堂山分校）
Li Zhang, Mingliang Wang, Mingxia Liu, Daoqiang Zhang
Deep learning has been recently used for the analysis of neuroimages, such as structural magnetic resonance imaging (MRI), functional MRI, and positron emission tomography (PET), and has achieved significant performance improvements over traditional machine learning in computer-aided diagnosis of brain disorders. This paper reviews the applications of deep learning methods for neuroimaging-based brain disorder analysis. We first provide a comprehensive overview of deep learning techniques and popular network architectures, by introducing various types of deep neural networks and recent developments. We then review deep learning methods for computer-aided analysis of four typical brain disorders, including Alzheimer's disease, Parkinson's disease, Autism spectrum disorder, and Schizophrenia, where the first two diseases are neurodegenerative disorders and the last two are neurodevelopmental and psychiatric disorders, respectively. More importantly, we discuss the limitations of existing studies and present possible future directions.
Self-Supervised Pretext Tasks for Alzheimer's Disease Classification using 3D Convolutional Neural Networks on Large-Scale Synthetic Neuroimaging Dataset
University of Pennsylvania（宾夕法尼亚大学）；Center for AI and Data Science for Integrated Diagnostics（综合诊断人工智能与数据科学中心）；Center for Biomedical Image Computing and Analytics（生物医学图像计算与分析中心）
Generative adversarial networks (GANs) are one powerful type of deep learning models that have been successfully utilized in numerous fields. They belong to a broader family called generative methods, which generate new data with a probabilistic model by learning sample distribution from real examples. In the clinical context, GANs have shown enhanced capabilities in capturing spatially complex, nonlinear, and potentially subtle disease effects compared to traditional generative methods. This review appraises the existing literature on the applications of GANs in imaging studies of various neu…
Brain-Aware Readout Layers in GNNs: Advancing Alzheimer's early Detection and Neuroimaging
AI Graduate School, Gwangju Institute of Science and Technology（光州科学技术院人工智能研究生院）；Seoul St. Mary’s Hospital, College of Medicine, The Catholic University of Korea（韩国天主教大学医学院首尔圣玛丽医院）；Yeouido St. Mary’s Hospital, College of Medicine, The Catholic University of Korea（韩国天主教大学医学院汝矣岛圣玛丽医院）
Deep learning requires large datasets for training (convolutional) networks with millions of parameters. In neuroimaging, there are few open datasets with more than 100 subjects, which makes it difficult to, for example, train a classifier to discriminate controls from diseased persons. Generative adversarial networks (GANs) can be used to synthesize data, but virtually all research is focused on 2D images. In medical imaging, and especially in neuroimaging, most datasets are 3D or 4D. Here we therefore present preliminary results showing that a 3D progressive growing GAN can be used to synthe…
MELAGE: A purely python based Neuroimaging software (Neonatal)
Biomedical Research and Innovation Institute of Cádiz (INiBICA)（加的斯生物医学研究与创新研究所（INiBICA））；Puerta del Mar University Hospital（海洋之门大学医院）；University of Cádiz（加的斯大学）
Uncovering Neuroimaging Biomarkers of Brain Tumor Surgery with AI-Driven Methods
Department of Communication Engineering University of Málaga（通信工程系马德里大学）；Department of Clinical Neurosciences University of Cambridge（临床神经科学系剑桥大学）；Department of Neuroscience University of Padova（神经科学系帕多瓦大学）；Department of Signal Theory, Telematics and Communications, University of Granada（信号理论、电信与通信系格拉纳达大学）；Department of Computer Science and Technology University of Cambridge（计算机科学与技术系剑桥大学）；Department of Psychiatry University of Cambridge（精神病学系剑桥大学）；NHS Foundation Trust United Kingdom（国家健康服务基金会信托英国）
Islamic Azad University（伊斯兰阿扎德大学）；K. N. Toosi University of Technology（K. N. 图西理工大学）；Ferdowsi University of Mashhad（马什哈德菲尔多西大学）；Semnan University（塞姆南大学）；Deakin University（迪肯大学）；Macquarie University（麦考瑞大学）；Dibrugarh University（迪布鲁格尔大学）
Accurate diagnosis of Autism Spectrum Disorder (ASD) followed by effective rehabilitation is essential for the management of this disorder. Artificial intelligence (AI) techniques can aid physicians to apply automatic diagnosis and rehabilitation procedures. AI techniques comprise traditional machine learning (ML) approaches and deep learning (DL) techniques. Conventional ML methods employ various feature extraction and classification techniques, but in DL, the process of feature extraction and classification is accomplished intelligently and integrally. DL methods for diagnosis of ASD have be…
Deep Learning for Brain Age Estimation: A Systematic Review
Indian Institute of Technology Indore（印度理工学院印多尔分校）；University of Michigan（密歇根大学）；University of Manitoba（曼尼托巴大学）；National Institute of Technology Silchar（锡尔查尔国家理工学院）；National Taipei University of Technology（国立台北科技大学）；Duke Kunshan University（昆山杜克大学）；University of Leicester（莱斯特大学）；TECNALIA（泰克纳利亚）；Basque Research and Technology Alliance (BRTA)（巴斯克研究与技术联盟）；University of the Basque Country (UPV/EHU)（巴斯克大学）
Tri-Institutional Center for Translational Research in Neuroimaging and Data Science（三机构神经影像与数据科学转化研究中心）；Georgia State（佐治亚州立大学）；Georgia Tech（佐治亚理工学院）；Emory（埃默里大学）；Borealis AI（Borealis人工智能公司）；Mila - Quebec AI Institute（米拉-魁北克人工智能研究所）；Apple Machine Learning Research（苹果机器学习研究院）；Université de Montréal（蒙特利尔大学）
Recent neuroimaging studies that focus on predicting brain disorders via modern machine learning approaches commonly include a single modality and rely on supervised over-parameterized models.However, a single modality provides only a limited view of the highly complex brain. Critically, supervised models in clinical settings lack accurate diagnostic labels for training. Coarse labels do not capture the long-tailed spectrum of brain disorder phenotypes, which leads to a loss of generalizability of the model that makes them less useful in diagnostic settings. This work presents a novel multi-sc…
Neuroimaging studies are often limited by the number of subjects and cognitive processes that can be feasibly interrogated. However, a rapidly growing number of neuroscientific studies have collectively accumulated an extensive wealth of results. Digesting this growing literature and obtaining novel insights remains to be a major challenge, since existing meta-analytic tools are constrained to keyword queries. In this paper, we present Text2Brain, an easy to use tool for synthesizing brain activation maps from open-ended text queries. Text2Brain was built on a transformer-based neural network …
Shenzhen Institutes of Advanced Technology, Chinese Academy of Sciences（中国科学院深圳先进技术研究院）
Cross-modal fusion of different types of neuroimaging data has shown great promise for predicting the progression of Alzheimer's Disease(AD). However, most existing methods applied in neuroimaging can not efficiently fuse the functional and structural information from multi-modal neuroimages. In this work, a novel cross-modal transformer generative adversarial network(CT-GAN) is proposed to fuse functional information contained in resting-state functional magnetic resonance imaging (rs-fMRI) and structural information contained in Diffusion Tensor Imaging (DTI). The developed bi-attention mech…
SPD Matrix Learning for Neuroimaging Analysis: Perspectives, Methods, and Challenges
Inria（法国国家信息与自动化研究所）；CEA（法国国家原子能与替代能源委员会）；Université Paris-Saclay（巴黎萨克雷大学）；Advanced Telecommunications Research Institute International (ATR)（国际先进电信研究机构）；RIKEN Artificial Intelligence Project（理化学研究所人工智能项目）；College of Computing and Data Science (CCDS) and Centre of AI in Medicine (C-AIM)（计算与数据科学学院和医学人工智能中心）
Massachusetts Eye and Ear（麻省眼耳医院）；Harvard Medical School（哈佛医学院）；Florida International University（佛罗里达国际大学）
Neuroimaging is essential in brain studies for the diagnosis and identification of disease, structure, and function of the brain in its healthy and disease states. Literature shows that there are advantages of multitasking with some deep learning (DL) schemes in challenging neuroimaging applications. This study examines the feasibility of using multitasking in three different applications, including tissue segmentation, cross-modality conversion, and bias-field correction. These applications reflect five different scenarios in which multitasking is explored and 280 training and testing session…
Deep learning algorithms for predicting neuroimaging data have shown considerable promise in various applications. Prior work has demonstrated that deep learning models that take advantage of the data's 3D structure can outperform standard machine learning on several learning tasks. However, most prior research in this area has focused on neuroimaging data from adults. Within the Adolescent Brain and Cognitive Development (ABCD) dataset, a large longitudinal development study, we examine structural MRI data to predict gender and identify gender-related changes in brain structure. Results demon…
Shenzhen Institute of Advanced Technology, Chinese Academy of Sciences（中国科学院深圳先进技术研究院）
Accurate diagnosis of autism spectrum disorder (ASD) based on neuroimaging data has significant implications, as extracting useful information from neuroimaging data for ASD detection is challenging. Even though machine learning techniques have been leveraged to improve the information extraction from neuroimaging data, the varying data quality caused by different meta-data conditions (i.e., data collection strategies) limits the effective information that can be extracted, thus leading to data-dependent predictive accuracies in ASD detection, which can be worse than random guess in some cases…
Unveiling Intractable Epileptogenic Brain Networks with Deep Learning Algorithms: A Novel and Comprehensive Framework for Scalable Seizure Prediction with Unimodal Neuroimaging Data in Pediatric Patients
We present BrainPainter, a software that automatically generates images of highlighted brain structures given a list of numbers corresponding to the output colours of each region. Compared to existing visualisation software (i.e. Freesurfer, SPM, 3D Slicer), BrainPainter has three key advantages: (1) it does not require the input data to be in a specialised format, allowing BrainPainter to be used in combination with any neuroimaging analysis tools, (2) it can visualise both cortical and subcortical structures and (3) it can be used to generate movies showing dynamic processes, e.g. propagatio…
Islamic Azad University（伊斯兰阿扎德大学）；Ferdowsi University of Mashhad（马什哈德菲尔多西大学）；Semnan University（塞姆南大学）；Qatar Biomedical Research Institute（卡塔尔生物医学研究所）；Hamad Bin Khalifa University（哈马德·本·哈利法大学）；University of Granada（格拉纳达大学）；Deakin University（迪肯大学）；University of Technology Sydney（悉尼科技大学）；University of Turku（图尔库大学）；Effat University（埃法特大学）；Ngee Ann Polytechnic（义安理工学院）；Asia University（亚洲大学）；Singapore University of Social Sciences（新加坡社会科学大学）
Autism spectrum disorder (ASD) is a brain condition characterized by diverse signs and symptoms that appear in early childhood. ASD is also associated with communication deficits and repetitive behavior in affected individuals. Various ASD detection methods have been developed, including neuroimaging modalities and psychological tests. Among these methods, magnetic resonance imaging (MRI) imaging modalities are of paramount importance to physicians. Clinicians rely on MRI modalities to diagnose ASD accurately. The MRI modalities are non-invasive methods that include functional (fMRI) and struc…
Neuroimaging of large populations is valuable to identify factors that promote or resist brain disease, and to assist diagnosis, subtyping, and prognosis. Data-driven models such as convolutional neural networks (CNNs) have increasingly been applied to brain images to perform diagnostic and prognostic tasks by learning robust features. Vision transformers (ViT) - a new class of deep learning architectures - have emerged in recent years as an alternative to CNNs for several computer vision applications. Here we tested variants of the ViT architecture for a range of desired neuroimaging downstre…
University of Texas Health Science Center at San Antonio（德克萨斯大学圣安东尼奥健康科学中心）；Charite – University Medicine Berlin（柏林夏里特医学院）；Humboldt-University Berlin（柏林洪堡大学）；Research Centre Jülich（于利希研究中心）；Heinrich-Heine University Düsseldorf（海因里希·海涅大学杜塞尔多夫分校）
Deep neural networks currently provide the most advanced and accurate machine learning models to distinguish between structural MRI scans of subjects with Alzheimer's disease and healthy controls. Unfortunately, the subtle brain alterations captured by these models are difficult to interpret because of the complexity of these multi-layer and non-linear models. Several heatmap methods have been proposed to address this issue and analyze the imaging patterns extracted from the deep neural networks, but no quantitative comparison between these methods has been carried out so far. In this work, we…
Over the last twenty five years, advances in the collection and analysis of fMRI data have enabled new insights into the brain basis of human health and disease. Individual behavioral variation can now be visualized at a neural level as patterns of connectivity among brain regions. Functional brain imaging is enhancing our understanding of clinical psychiatric disorders by revealing ties between regional and network abnormalities and psychiatric symptoms. Initial success in this arena has recently motivated collection of larger datasets which are needed to leverage fMRI to generate brain-based…
Pipeline-Invariant Representation Learning for Neuroimaging
Georgia State University（佐治亚州立大学）；Georgia Institute of Technology（佐治亚理工学院）；Emory University（埃默里大学）；Center for Translational Research in Neuroimaging and Data Science (TReNDS)（神经影像与数据科学转化研究中心（TReNDS））；Child Mind Institute（儿童心理研究所）
Magnetic Resonance Imaging (MRI) of the brain can come in the form of different modalities such as T1-weighted and Fluid Attenuated Inversion Recovery (FLAIR) which has been used to investigate a wide range of neurological disorders. Current state-of-the-art models for brain tissue segmentation and disease classification require multiple modalities for training and inference. However, the acquisition of all of these modalities are expensive, time-consuming, inconvenient and the required modalities are often not available. As a result, these datasets contain large amounts of \emph{unpaired} dat…
Advances in imaging technologies, combined with inexpensive storage, have led to an explosion in the volume of publicly available neuroimaging datasets. Effective analyses of these images hold the potential for uncovering mechanisms that govern functioning of the human brain, and understanding various neurological diseases and disorders. The potential significance of these studies notwithstanding, a growing concern relates to the protection of privacy and confidentiality of subjects who participate in these studies. In this paper, we present a de-anonymization attack rooted in the innate uniqu…
University of Southern California（南加州大学）；Information Sciences Institute（信息科学研究所）；Mark and Mary Stevens Institute for Neuroimaging and Informatics（马克与玛丽·史蒂文斯神经影像与信息学研究所）；Keck School of Medicine（凯克医学院）
Deep Learning for neuroimaging data is a promising but challenging direction. The high dimensionality of 3D MRI scans makes this endeavor compute and data-intensive. Most conventional 3D neuroimaging methods use 3D-CNN-based architectures with a large number of parameters and require more time and data to train. Recently, 2D-slice-based models have received increasing attention as they have fewer parameters and may require fewer samples to achieve comparable performance. In this paper, we propose a new architecture for BrainAGE prediction. The proposed architecture works by encoding each 2D sl…
German Center for Neurodegenerative Diseases (DZNE)（德国神经退行性疾病中心（DZNE））；A.A. Martinos Center for Biomedical Imaging, Massachusetts General Hospital（麻省总医院A.A.马丁诺斯生物医学成像中心）；Harvard Medical School（哈佛医学院）；MIT（麻省理工学院）
Traditional neuroimage analysis pipelines involve computationally intensive, time-consuming optimization steps, and thus, do not scale well to large cohort studies with thousands or tens of thousands of individuals. In this work we propose a fast and accurate deep learning based neuroimaging pipeline for the automated processing of structural human brain MRI scans, replicating FreeSurfer's anatomical segmentation including surface reconstruction and cortical parcellation. To this end, we introduce an advanced deep learning architecture capable of whole brain segmentation into 95 classes. The n…
The development of automated tools for brain morphometric analysis in infants has lagged significantly behind analogous tools for adults. This gap reflects the greater challenges in this domain due to: 1) a smaller-scaled region of interest, 2) increased motion corruption, 3) regional changes in geometry due to heterochronous growth, and 4) regional variations in contrast properties corresponding to ongoing myelination and other maturation processes. Nevertheless, there is a great need for automated image-processing tools to quantify differences between infant groups and other individuals, bec…
Information Sciences Institute, University of Southern California（南加州大学信息科学研究所）；Mark and Mary Stevens Institute for Neuroimaging and Informatics, Keck School of Medicine, University of Southern California（南加州大学凯克医学院马克与玛丽·史蒂文斯神经影像与信息学研究所）；University of Southern California（南加州大学）
Ensuring the privacy of research participants is vital, even more so in healthcare environments. Deep learning approaches to neuroimaging require large datasets, and this often necessitates sharing data between multiple sites, which is antithetical to the privacy objectives. Federated learning is a commonly proposed solution to this problem. It circumvents the need for data sharing by sharing parameters during the training process. However, we demonstrate that allowing access to parameters may leak private information even if data is never directly shared. In particular, we show that it is pos…
University of Macau（澳门大学）；Shenzhen Institutes of Advanced Technology, Chinese Academy of Sciences（中国科学院深圳先进技术研究院）；Shenzhen University（深圳大学）
Brain network analysis is essential for diagnosing and intervention for Alzheimer's disease (AD). However, previous research relied primarily on specific time-consuming and subjective toolkits. Only few tools can obtain the structural brain networks from brain diffusion tensor images (DTI). In this paper, we propose a diffusion based end-to-end brain network generative model Brain Diffuser that directly shapes the structural brain networks from DTI. Compared to existing toolkits, Brain Diffuser exploits more structural connectivity features and disease-related information by analyzing disparit…
