精神分裂症（SZ）是一种严重的精神疾病。其发病机制尚未完全明确，使得精准治疗患者变得困难。由于人脑复杂的非欧几里得网络结构，从大脑网络中学习关键信息仍然存在困难。为了有效捕获大脑神经网络的拓扑信息，提出了一种基于稀疏交互机制的新型多模态图注意力网络（Multi-SIGATnet）用于SZ分类。首先，将结构和功能信息融合为多模态数据，以获取SZ患者更全面、更丰富的特征。随后，提出稀疏交互机制以有效提取显著特征并增强特征表示能力。通过基于非对称卷积网络增强特征信息间的强连接并削弱弱连接，捕获了高阶交互特征。此外，设计了稀疏学习策略以过滤冗余连接，从而提升模型性能。最后，根据高阶脑网络的拓扑特征和连接权重约束更新局部和全局特征，将特征投影到分类目标空间进行疾病分类。在Center for Biomedical Research Excellence（COBRE）和University of California Los Angeles（UCLA）数据集上验证了模型的有效性，分别实现了81.9%和75.8%的平均准确率，比图注意力网络（GAT）方法分别高出4.6%和5.5%。实验表明，Multi-SIGATnet方法在识别SZ方面表现出良好的性能。
英文摘要：
Schizophrenia is a serious psychiatric disorder. Its pathogenesis is not completely clear, making it difficult to treat patients precisely. Because of the complicated non-Euclidean network structure of the human brain, learning critical information from brain networks remains difficult. To effectively capture the topological information of brain neural networks, a novel multimodal graph attention network based on sparse interaction mechanism (Multi-SIGATnet) was proposed for SZ classification was proposed for SZ classification. Firstly, structural and functional information were fused into multimodal data to obtain more comprehensive and abundant features for patients with SZ. Subsequently, a sparse interaction mechanism was proposed to effectively extract salient features and enhance the feature representation capability. By enhancing the strong connections and weakening the weak connections between feature information based on an asymmetric convolutional network, high-order interactive features were captured. Moreover, sparse learning strategies were designed to filter out redundant connections to improve model performance. Finally, local and global features were updated in accordance with the topological features and connection weight constraints of the higher-order brain network, the features being projected to the classification target space for disorder classification. The effectiveness of the model is verified on the Center for Biomedical Research Excellence (COBRE) and University of California Los Angeles (UCLA) datasets, achieving 81.9\% and 75.8\% average accuracy, respectively, 4.6\% and 5.5\% higher than the graph attention network (GAT) method. Experiments showed that the Multi-SIGATnet method exhibited good performance in identifying SZ.
Vision Transformer (ViT) is a pioneering deep learning framework that can address real-world computer vision issues, such as image classification and object recognition. Importantly, ViTs are proven to outperform traditional deep learning models, such as convolutional neural networks (CNNs). Relatively recently, a number of ViT mutations have been transplanted into the field of medical imaging, thereby resolving a variety of critical classification and segmentation challenges, especially in terms of brain imaging data. In this work, we provide a novel multimodal deep learning pipeline, MultiCr…
Institute of Mental Health, Singapore（新加坡精神卫生研究院）；National University of Singapore（新加坡国立大学）；Nanyang Technological University（南洋理工大学）；Duke-NUS Medical School（杜克-新加坡国立大学医学院）
Convolutional Neural Network (CNN) has been successfully applied on classification of both natural images and medical images but not yet been applied to differentiating patients with schizophrenia from healthy controls. Given the subtle, mixed, and sparsely distributed brain atrophy patterns of schizophrenia, the capability of automatic feature learning makes CNN a powerful tool for classifying schizophrenia from controls as it removes the subjectivity in selecting relevant spatial features. To examine the feasibility of applying CNN to classification of schizophrenia and controls based on str…
National Institute of Technology Silchar（西尔查尔国家理工学院）；Indian Institute of Technology, Indore（印多尔印度理工学院）
Structural alterations have been thoroughly investigated in the brain during the early onset of schizophrenia (SCZ) with the development of neuroimaging methods. The objective of the paper is an efficient classification of SCZ in 2 different classes: Cognitive Normal (CN), and SCZ using magnetic resonance imaging (MRI) images. This paper proposed a lightweight 3D convolutional neural network (CNN) based framework for SCZ diagnosis using MRI images. In the proposed model, lightweight 3D CNN is used to extract both spatial and spectral features simultaneously from 3D volume MRI scans, and classi…
An Interpretable Cross-Attentive Multi-modal MRI Fusion Framework for Schizophrenia Diagnosis
Tulane University（杜兰大学）；Tri-Institutional Center for Translational Research in Neuroimaging and Data Science (TReNDS)（跨机构神经成像与数据科学转化研究中心（TReNDS））；Georgia State University（佐治亚州立大学）；Georgia Institute of Technology（佐治亚理工学院）；Emory University（埃默里大学）
Nanyang Technological University（南洋理工大学）；NTU Institute for Health Technologies（南洋理工大学健康技术研究所）；National University of Singapore（新加坡国立大学）；Yong Loo Lin School of Medicine（杨潞龄医学院）；Institute of Mental Health (IMH)（心理健康研究所（IMH））
Deep learning has been successfully applied to recognizing both natural images and medical images. However, there remains a gap in recognizing 3D neuroimaging data, especially for psychiatric diseases such as schizophrenia and depression that have no visible alteration in specific slices. In this study, we propose to process the 3D data by a 2+1D framework so that we can exploit the powerful deep 2D Convolutional Neural Network (CNN) networks pre-trained on the huge ImageNet dataset for 3D neuroimaging recognition. Specifically, 3D volumes of Magnetic Resonance Imaging (MRI) metrics (grey matt…
Columbia University（哥伦比亚大学）；The Mortimer B. Zuckerman Mind Brain Behavior Institute（莫蒂默·B·扎克曼心智大脑行为研究所）；The Taub Institute for Research on Alzheimer's Disease and the Aging Brain（陶布阿尔茨海默病与衰老大脑研究中心）；Jericho High School（杰里科高中）
Schizophrenia is a chronic neuropsychiatric disorder that causes distinct structural alterations within the brain. We hypothesize that deep learning applied to a structural neuroimaging dataset could detect disease-related alteration and improve classification and diagnostic accuracy. We tested this hypothesis using a single, widely available, and conventional T1-weighted MRI scan, from which we extracted the 3D whole-brain structure using standard post-processing methods. A deep learning model was then developed, optimized, and evaluated on three open datasets with T1-weighted MRI scans of pa…
Université Paris-Saclay（巴黎萨克雷大学）；Télécom Paris（巴黎电信学院）；Johannes-Gutenberg Univ. of Mainz（美因茨约翰内斯·古腾堡大学）；University of Milan（米兰大学）；Université Grenoble Alpes（格勒诺布尔阿尔卑斯大学）；CHU Grenoble Alpe（格勒诺布尔阿尔卑斯大学中心医院）；Centre for Neuroimaging & Cognitive Genomics (NICOG)（神经影像与认知基因组学中心）
Traditional supervised learning with deep neural networks requires a tremendous amount of labelled data to converge to a good solution. For 3D medical images, it is often impractical to build a large homogeneous annotated dataset for a specific pathology. Self-supervised methods offer a new way to learn a representation of the images in an unsupervised manner with a neural network. In particular, contrastive learning has shown great promises by (almost) matching the performance of fully-supervised CNN on vision tasks. Nonetheless, this method does not take advantage of available meta-data, suc…
DDEvENet: Evidence-based Ensemble Learning for Uncertainty-aware Brain Parcellation Using Diffusion MRI
University of Electronic Science and Technology of China（电子科技大学）；The First Affiliated Hospital, Sun Yat-sen University（中山大学附属第一医院）；The Second Affiliated Hospital, Zhejiang University School of Medicine（浙江大学医学院附属第二医院）；Nanjing University of Science and Technology（南京理工大学）；East China University of Science and Technology（华东理工大学）；Harvard Medical School（哈佛医学院）；School of Mathematical Sciences, University of Tel Aviv（特拉维夫大学数学科学学院）
Rutgers University（罗格斯大学）；Massachusetts Institute of Technology（麻省理工学院）；University of Alberta（阿尔伯塔大学）；IBM T.J. Watson Research Center（IBM托马斯·J·沃森研究中心）
Smart systems that can accurately diagnose patients with mental disorders and identify effective treatments based on brain functional imaging data are of great applicability and are gaining much attention. Most previous machine learning studies use hand-designed features, such as functional connectivity, which does not maintain the potential useful information in the spatial relationship between brain regions and the temporal profile of the signal in each region. Here we propose a new method based on recurrent-convolutional neural networks to automatically learn useful representations from seg…
Multi-Dimension-Embedding-Aware Modality Fusion Transformer for Psychiatric Disorder Clasification
Zhejiang University（浙江大学）；JD Health International Inc.（京东健康国际有限公司）；Beijing Huilongguan Hospital（北京回龙观医院）；Peking University Huilongguan Clinical Medical School（北京大学回龙观临床医学院）
School of Information Technologies, University of Sydney（悉尼大学信息技术学院）；School of Medicine, University of Sydney（悉尼大学医学院）；Centre for Translational Datascience, University of Sydney（悉尼大学转化数据科学中心）；Brain Dynamics Centre, Westmead Millennium Institute（西mead千年研究所脑动力学中心）
Magnetic Resonance Imaging (MRI) of the brain has been used to investigate a wide range of neurological disorders, but data acquisition can be expensive, time-consuming, and inconvenient. Multi-site studies present a valuable opportunity to advance research by pooling data in order to increase sensitivity and statistical power. However images derived from MRI are susceptible to both obvious and non-obvious differences between sites which can introduce bias and subject variance, and so reduce statistical power. To rectify these differences, we propose a data driven approach using a deep learnin…
College of Computer Science and Technology, Nanjing Forestry University（南京林业大学计算机科学与技术学院）；College of Computer Science and Technology, Nanjing University of Aeronautics and Astronautics（南京航空航天大学计算机科学与技术学院）；University of North Carolina at Chapel Hill（北卡罗来纳大学教堂山分校）
Deep learning has been recently used for the analysis of neuroimages, such as structural magnetic resonance imaging (MRI), functional MRI, and positron emission tomography (PET), and has achieved significant performance improvements over traditional machine learning in computer-aided diagnosis of brain disorders. This paper reviews the applications of deep learning methods for neuroimaging-based brain disorder analysis. We first provide a comprehensive overview of deep learning techniques and popular network architectures, by introducing various types of deep neural networks and recent develop…
The Mind Research Network（心智研究网络）；University of New Mexico（新墨西哥大学）
Large scale studies of group differences in healthy controls and patients and screenings for early stage disease prevention programs require processing and analysis of extensive multisubject datasets. Complexity of the task increases even further when segmenting structural MRI of the brain into an atlas with more than 50 regions. Current automatic approaches are time-consuming and hardly scalable; they often involve many error prone intermediate steps and don't utilize other available modalities. To alleviate these problems, we propose a feedforward fully convolutional neural network trained o…
