TractGraphFormer: Anatomically Informed Hybrid Graph CNN-Transformer Network for Interpretable Sex and Age Prediction from Diffusion MRI Tractography
Brigham and Women's Hospital, Harvard Medical School（哈佛医学院布莱根妇女医院）；University of Electronic Science and Technology of China（电子科技大学）；Beth Israel Deaconess Medical Center, Harvard Medical School（哈佛医学院贝斯以色列女执事医疗中心）；The University of Sydney（悉尼大学）；University of New South Wales（新南威尔士大学）；Boston University School of Medicine（波士顿大学医学院）；Massachusetts General Hospital, Harvard Medical School（哈佛医学院麻省总医院）
Yuqian Chen, Fan Zhang, Meng Wang, Leo R. Zekelman, Suheyla Cetin-Karayumak, Tengfei Xue, Chaoyi Zhang, Yang Song, Jarrett Rushmore, Nikos Makris, Yogesh Rathi,… 展开作者
Yuqian Chen, Fan Zhang, Meng Wang, Leo R. Zekelman, Suheyla Cetin-Karayumak, Tengfei Xue, Chaoyi Zhang, Yang Song, Jarrett Rushmore, Nikos Makris, Yogesh Rathi, Weidong Cai, Lauren J. O'Donnell
The relationship between brain connections and non-imaging phenotypes is increasingly studied using deep neural networks. However, the local and global properties of brain white matter networks are often overlooked in convolutional network design. We introduce TractGraphFormer, a hybrid Graph CNN-Transformer deep learning framework tailored for diffusion MRI tractography. This model leverages local anatomical characteristics and global feature dependencies of white matter structures. The Graph CNN module captures white matter geometry and grey matter connectivity to aggregate local features from anatomically similar white matter connections, while the Transformer module uses self-attention to enhance global information learning. Additionally, TractGraphFormer includes an attention module for interpreting predictive white matter connections. We apply TractGraphFormer to tasks of sex and age prediction. TractGraphFormer shows strong performance in large datasets of children (n=9345) and young adults (n=1065). Overall, our approach suggests that widespread connections in the WM are predictive of the sex and age of an individual. For each prediction task, consistent predictive anatomical tracts are identified across the two datasets. The proposed approach highlights the potential of integrating local anatomical information and global feature dependencies to improve prediction performance in machine learning with diffusion MRI tractography.
Harvard Medical School（哈佛医学院）；The University of Sydney（悉尼大学）；The University of New South Wales（新南威尔士大学）
The structure and variability of the brain's connections can be investigated via prediction of non-imaging phenotypes using neural networks. However, known neuroanatomical relationships between input features are generally ignored in network design. We propose TractGraphCNN, a novel, anatomically informed graph CNN framework for machine learning tasks using diffusion MRI tractography. An EdgeConv module aggregates features from anatomically similar white matter connections indicated by graph edges, and an attention module enables interpretation of predictive white matter tracts. Results in a s…
TractCloud-FOV: Deep Learning-based Robust Tractography Parcellation in Diffusion MRI with Incomplete Field of View
Harvard Medical School（哈佛医学院）；Harvard University（哈佛大学）；The University of Sydney（悉尼大学）；Massachusetts General Hospital（麻省总医院）；University of Electronic Science and Technology of China（电子科技大学）；Harvard-MIT Health Sciences and Technology（哈佛-麻省理工健康科学与技术部）
Indian Institute of Technology, Mandi（印度理工学院曼迪分校）；Indian Institute of Technology, Jodhpur（印度理工学院焦特布尔分校）
The segregation of brain fiber tractography data into distinct and anatomically meaningful clusters can help to comprehend the complex brain structure and early investigation and management of various neural disorders. We propose a novel stacked bidirectional long short-term memory(LSTM) based segmentation network, (BrainSegNet) for human brain fiber tractography data classification. We perform a two-level hierarchical classification a) White vs Grey matter (Macro) and b) White matter clusters (Micro). BrainSegNet is trained over three brain tractography data having over 250,000 fibers each. O…
AGFS-Tractometry: A Novel Atlas-Guided Fine-Scale Tractometry Approach for Enhanced Along-Tract Group Statistical Comparison Using Diffusion MRI Tractography
University of Electronic Science and Technology of China（电子科技大学）；Brigham and Women’s Hospital, Harvard Medical School（哈佛医学院布里奇曼妇女医院）；Boston University（波士顿大学）
Indian Institute of Technology Jodhpur（印度理工学院焦特布尔分校）；Indian Institute of Technology Mandi（印度理工学院曼迪分校）
In this paper, we propose a novel deep learning architecture combining stacked Bi-directional LSTM and LSTMs with the Siamese network architecture for segmentation of brain fibers, obtained from tractography data, into anatomically meaningful clusters. The proposed network learns the structural difference between fibers of different classes, which enables it to classify fibers with high accuracy. Importantly, capturing such deep inter and intra class structural relationship also ensures that the segmentation is robust to relative rotation among test and training data, hence can be used with un…
A Novel Streamline-based diffusion MRI Tractography Registration Method with Probabilistic Keypoint Detection
University of Electronic Science and Technology of China（电子科技大学）；Nanjing University of Science and Technology（南京理工大学）；Harvard Medical School（哈佛医学院）；Brigham and Women’s Hospital（布莱根和妇女医院）
School of Computer Science, University of Sydney（悉尼大学计算机科学学院）；School of Computer Science and Engineering, University of New South Wales（新南威尔士大学计算机科学与工程学院）；Center for Morphometric Analysis, Massachusetts General Hospital（麻省总医院形态测量分析中心）
White matter parcellation classifies tractography streamlines into clusters or anatomically meaningful tracts to enable quantification and visualization. Most parcellation methods focus on the deep white matter (DWM), while fewer methods address the superficial white matter (SWM) due to its complexity. We propose a deep-learning-based framework, Superficial White Matter Analysis (SupWMA), that performs an efficient and consistent parcellation of 198 SWM clusters from whole-brain tractography. A point-cloud-based network is modified for our SWM parcellation task, and supervised contrastive lear…
Hawkes Institute and Department of Computer Science, University College London, UK（霍克斯研究所和计算机科学系，伦敦大学学院，英国）；Department of Maths and Computer Science, University of Catania, Italy（数学和计算机科学系，卡塔尼亚大学，意大利）；AI Centre and Department of Computer Science, University College London, UK（人工智能中心和计算机科学系，伦敦大学学院，英国）
DDTracking: A Deep Generative Framework for Diffusion MRI Tractography with Streamline Local-Global Spatiotemporal Modeling
University of Electronic Science and Technology of China（电子科技大学）；Nanjing University of Science and Technology（南京理工大学）；Brigham and Women’s Hospital（布里奇沃特医院）；Harvard Medical School（哈佛医学院）
Brigham and Women’s Hospital（布莱根妇女医院）；Harvard Medical School（哈佛医学院）；University of Sydney（悉尼大学）；University of New South Wales（新南威尔士大学）；Massachusetts General Hospital（麻省总医院）
Diffusion MRI tractography is an advanced imaging technique that enables in vivo mapping of the brain's white matter connections. White matter parcellation classifies tractography streamlines into clusters or anatomically meaningful tracts. It enables quantification and visualization of whole-brain tractography. Currently, most parcellation methods focus on the deep white matter (DWM), whereas fewer methods address the superficial white matter (SWM) due to its complexity. We propose a novel two-stage deep-learning-based framework, Superficial White Matter Analysis (SupWMA), that performs an ef…
Harvard Medical School（哈佛医学院）；The University of Sydney（悉尼大学）；The University of New South Wales（新南威尔士大学）
White matter fiber clustering (WMFC) enables parcellation of white matter tractography for applications such as disease classification and anatomical tract segmentation. However, the lack of ground truth and the ambiguity of fiber data (the points along a fiber can equivalently be represented in forward or reverse order) pose challenges to this task. We propose a novel WMFC framework based on unsupervised deep learning. We solve the unsupervised clustering problem as a self-supervised learning task. Specifically, we use a convolutional neural network to learn embeddings of input fibers, using …
A Novel Dual-Stream Framework for dMRI Tractography Streamline Classification with Joint dMRI and fMRI Data
University of Electronic Science and Technology of China（电子科技大学）；Zhejiang University of Technology（浙江工业大学）；University of Tel Aviv（特拉维夫大学）；Harvard Medical School（哈佛医学院）
Université de Sherbrooke（舍布鲁克大学）；Université Bordeaux（波尔多大学）；CNRS（法国国家科学研究中心）；CEA（法国原子能和替代能源委员会）；Imeka Solutions inc.（Imeka解决方案公司）
Current tractography methods use the local orientation information to propagate streamlines from seed locations. Many such seeds provide streamlines that stop prematurely or fail to map the true white matter pathways because some bundles are "harder-to-track" than others. This results in tractography reconstructions with poor white and gray matter spatial coverage. In this work, we propose a generative, autoencoder-based method, named GESTA (Generative Sampling in Bundle Tractography using Autoencoders), that produces streamlines achieving better spatial coverage. Compared to other deep learni…
Detailed delineation of the fetal brain in diffusion MRI via multi-task learning
Harvard Medical School（哈佛医学院）；Elmhurst Hospital Center（埃尔姆赫斯特医院中心）；Icahn School of Medicine at Mount Sinai（西奈山伊坎医学院）；Massachusetts General Hospital（麻省总医院）；University of California Irvine（加州大学欧文分校）
German Cancer Research Center (DKFZ)（德国癌症研究中心）；University of Heidelberg（海德堡大学）；Heidelberg University Hospital（海德堡大学医院）
The individual course of white matter fiber tracts is an important key for analysis of white matter characteristics in healthy and diseased brains. Uniquely, diffusion-weighted MRI tractography in combination with region-based or clustering-based selection of streamlines allows for the in-vivo delineation and analysis of anatomically well known tracts. This, however, currently requires complex, computationally intensive and tedious-to-set-up processing pipelines. TractSeg is a novel convolutional neural network-based approach that directly segments tracts in the field of fiber orientation dist…
FIESTA: Autoencoders for accurate fiber segmentation in tractography
Université de Sherbrooke, Canada（舍布鲁克大学，加拿大）；Sherbrooke Connectivity Imaging Lab (SCIL)（舍布鲁克连接成像实验室（SCIL））；Videos & Images Theory and Analytics Lab (VITAL)（视频与图像理论及分析实验室（VITAL））；Department of Computer Science（计算机科学系）；Department of Radiology（放射科）；Brigham and Women’s Hospital（布莱根妇女医院）；Mass General Brigham/Harvard Medical School, USA（麻省总医院布莱根联合/哈佛医学院，美国）；Centre de Calcul Scientifique（科学计算中心）；Medical Imaging and Neuroinformatic (MINi) Lab（医学成像与神经信息学（MINi）实验室）；Groupe d’Imagerie Neurofonctionnelle (GIN)（神经功能成像小组（GIN））；CNRS, CEA, IMN, GIN, UMR 5293, F-33000 Bordeaux（法国国家科学研究中心、法国原子能委员会、神经影像研究所、神经功能成像小组、5293号联合研究单位，波尔多）；Université de Bordeaux, France（波尔多大学，法国）；Pharma Research and Early Development（制药研究与早期开发）；Neuroscience and Rare Diseases（神经科学与罕见病）；Roche Innovation Center Basel（巴塞尔罗氏创新中心）；F. Hoffmann-La Roche Ltd., Basel, Switzerland（F.霍夫曼-罗氏有限公司，巴塞尔，瑞士）
Assessing Streamline Plausibility Through Randomized Iterative Spherical-Deconvolution Informed Tractogram Filtering
Saarland University（萨尔大学）；Krembil Research Institute（克伦比尔研究所）；Toronto Western Hospital（多伦多西部医院）；University Health Network（大学健康网络）；KTH Royal Institute of Technology（KTH皇家理工学院）
