Sub-cortical brain structure segmentation using F-CNN's
CentraleSupelec（巴黎中央高等电力学院）；Inria（法国国家信息与自动化研究所）；Universite Paris-Saclay（巴黎萨克雷大学）；Polytechnique Montreal（蒙特利尔理工学院）；University of Montreal（蒙特利尔大学）；Sainte-Justine Hospital Research Center（圣朱斯汀医院研究中心）
Mahsa Shakeri, Stavros Tsogkas, Enzo Ferrante, Sarah Lippe, Samuel Kadoury, Nikos Paragios, Iasonas Kokkinos
备注：
ISBI 2016: International Symposium on Biomedical Imaging, Apr 2016, Prague, Czech Republic
In this paper we propose a deep learning approach for segmenting sub-cortical structures of the human brain in Magnetic Resonance (MR) image data. We draw inspiration from a state-of-the-art Fully-Convolutional Neural Network (F-CNN) architecture for semantic segmentation of objects in natural images, and adapt it to our task. Unlike previous CNN-based methods that operate on image patches, our model is applied on a full blown 2D image, without any alignment or registration steps at testing time. We further improve segmentation results by interpreting the CNN output as potentials of a Markov Random Field (MRF), whose topology corresponds to a volumetric grid. Alpha-expansion is used to perform approximate inference imposing spatial volumetric homogeneity to the CNN priors. We compare the performance of the proposed pipeline with a similar system using Random Forest-based priors, as well as state-of-art segmentation algorithms, and show promising results on two different brain MRI datasets.
Harvard University（哈佛大学）；Princeton University（普林斯顿大学）；University of Chicago（芝加哥大学）；RWTH Aachen University（亚琛工业大学）；Boston University（波士顿大学）；Universidade do Vale do Rio dos Sinos（南里奥格兰德河谷大学）；Argonne National Laboratory（阿贡国家实验室）；Donostia International Physics Center（多诺斯蒂亚国际物理中心）；University of the Basque Country (UPV/EHU)（巴斯克大学（UPV/EHU））；Ikerbasque, Basque Foundation for Science（巴斯克科学基金会（Ikerbasque））
Electron microscopy (EM) enables the reconstruction of neural circuits at the level of individual synapses, which has been transformative for scientific discoveries. However, due to the complex morphology, an accurate reconstruction of cortical axons has become a major challenge. Worse still, there is no publicly available large-scale EM dataset from the cortex that provides dense ground truth segmentation for axons, making it difficult to develop and evaluate large-scale axon reconstruction methods. To address this, we introduce the AxonEM dataset, which consists of two 30x30x30 um^3 EM image…
NeuroNet is a deep convolutional neural network mimicking multiple popular and state-of-the-art brain segmentation tools including FSL, SPM, and MALPEM. The network is trained on 5,000 T1-weighted brain MRI scans from the UK Biobank Imaging Study that have been automatically segmented into brain tissue and cortical and sub-cortical structures using the standard neuroimaging pipelines. Training a single model from these complementary and partially overlapping label maps yields a new powerful "all-in-one", multi-output segmentation tool. The processing time for a single subject is reduced by an …
The human brain cortical layer has a convoluted morphology that is unique to each individual. Characterization of the cortical morphology is necessary in longitudinal studies of structural brain change, as well as in discriminating individuals in health and disease. A method for encoding the cortical morphology in the form of a graph is presented. The design of graphs that encode the global cerebral hemisphere cortices as well as localized cortical regions is proposed. Spectral metrics derived from these graphs are then studied and proposed as descriptors of cortical morphology. As proof-of-co…
Sahand University of Technology（萨汉德工业大学）；Lund University（隆德大学）
The human brain cortical layer has a convoluted morphology that is unique to each individual. Characterization of the cortical morphology is necessary in longitudinal studies of structural brain change, as well as in discriminating individuals in health and disease. A method for encoding the cortical morphology in the form of a graph is presented. The design of graphs that encode the global cerebral hemisphere cortices as well as localized cortical regions is proposed. Spectral features of these graphs are then studied and proposed as descriptors of cortical morphology. As proof-of-concept of …
Adaptive Cortically Constrained EEG-Vision Alignment for Zero-Shot Brain-to-Image Retrieval
Chongqing University of Posts and Telecommunications（重庆邮电大学）；Shanghai Jiaotong University（上海交通大学）；Chongqing Normal University（重庆师范大学）；South China University of Technology（华南理工大学）
Northwestern Polytechnical University（西北工业大学）；University of Texas at Arlington（德克萨斯大学阿灵顿分校）；University of Georgia（佐治亚大学）
Since gyri and sulci, two basic anatomical building blocks of cortical folding patterns, were suggested to bear different functional roles, a precise mapping from brain function to gyro-sulcal patterns can provide profound insights into both biological and artificial neural networks. However, there lacks a generic theory and effective computational model so far, due to the highly nonlinear relation between them, huge inter-individual variabilities and a sophisticated description of brain function regions/networks distribution as mosaics, such that spatial patterning of them has not been consid…
Neuronal cell bodies mostly reside in the cerebral cortex. The study of this thin and highly convoluted surface is essential for understanding how the brain works. The analysis of surface data is, however, challenging due to the high variability of the cortical geometry. This paper presents a novel approach for learning and exploiting surface data directly across surface domains. Current approaches rely on geometrical simplifications, such as spherical inflations, a popular but costly process. For instance, the widely used FreeSurfer takes about 3 hours to parcellate brain surfaces on a standa…
Rheinische Friedrich-Wilhelms-Universität Bonn（波恩大学）；Hôpitaux Universitaires de Genève（日内瓦大学医院）
Brain shift, i.e. the change in configuration of the brain after opening the dura mater, is a key problem in neuronavigation. We present an approach to co-register intra-operative microscope images with pre-operative MRI to adapt and optimize intra-operative neuronavigation. The tools are a robust classification of sulci on MRI extracted cortical surfaces, guided user marking of most prominent sulci on a microscope image, and the actual variational registration method with a fidelity energy for 3D deformations of the cortical surface combined with a higher order, linear elastica type prior ene…
Institute of Computer Vision and Robotics（计算机视觉与机器人研究所）；University of Girona（赫罗纳大学）
Sub-cortical brain structure segmentation in Magnetic Resonance Images (MRI) has attracted the interest of the research community for a long time because morphological changes in these structures are related to different neurodegenerative disorders. However, manual segmentation of these structures can be tedious and prone to variability, highlighting the need for robust automated segmentation methods. In this paper, we present a novel convolutional neural network based approach for accurate segmentation of the sub-cortical brain structures that combines both convolutional and prior spatial fea…
Surface-volume self-supervised representation learning of brain MRI for genetic discovery
The University of Texas Health Science Center at Houston（德克萨斯大学休斯顿健康科学中心）；Yale School of Medicine（耶鲁大学医学院）；Texas A&M University（德克萨斯A&M大学）；Stanford University School of Medicine（斯坦福大学医学院）
School of Computing, Informatics and Decision Systems Engineering, Arizona State University（亚利桑那州立大学计算、信息与决策系统工程学院）
A large number of surface-based analyses on brain imaging data adopt some specific brain atlases to better assess structural and functional changes in one or more brain regions. In these analyses, it is necessary to obtain an anatomically correct surface parcellation scheme in an individual brain by referring to the given atlas. Traditional ways to accomplish this goal are through a designed surface-based registration or hand-crafted surface features, although both of them are time-consuming. A recent deep learning approach depends on a regular spherical parameterization of the mesh, which is …
Fetal cortical plate segmentation is essential in quantitative analysis of fetal brain maturation and cortical folding. Manual segmentation of the cortical plate, or manual refinement of automatic segmentations is tedious and time-consuming. Automatic segmentation of the cortical plate, on the other hand, is challenged by the relatively low resolution of the reconstructed fetal brain MRI scans compared to the thin structure of the cortical plate, partial voluming, and the wide range of variations in the morphology of the cortical plate as the brain matures during gestation. To reduce the burde…
Explicitly Modeling Pre-Cortical Vision with a Neuro-Inspired Front-End Improves CNN Robustness
IST Técnico Lisboa（里斯本理工学院）；Universidade de Lisboa（里斯本大学）；INESC-ID Lisboa（里斯本INESC-ID研究所）；Champalimaud Foundation（尚帕利莫基金会）；Champalimaud Clinical Center（尚帕利莫临床中心）
The Chinese University of Hong Kong（香港中文大学）；Massachusetts Institute of Technology（麻省理工学院）
In medical imaging, surface registration is extensively used for performing systematic comparisons between anatomical structures, with a prime example being the highly convoluted brain cortical surfaces. To obtain a meaningful registration, a common approach is to identify prominent features on the surfaces and establish a low-distortion mapping between them with the feature correspondence encoded as landmark constraints. Prior registration works have primarily focused on using manually labeled landmarks and solving highly nonlinear optimization problems, which are time-consuming and hence hin…
Technical University of Munich（慕尼黑工业大学）；Ludwig-Maximilians-University Munich（慕尼黑路德维希-马克西米利安大学）
The reconstruction of cortical surfaces from brain magnetic resonance imaging (MRI) scans is essential for quantitative analyses of cortical thickness and sulcal morphology. Although traditional and deep learning-based algorithmic pipelines exist for this purpose, they have two major drawbacks: lengthy runtimes of multiple hours (traditional) or intricate post-processing, such as mesh extraction and topology correction (deep learning-based). In this work, we address both of these issues and propose Vox2Cortex, a deep learning-based algorithm that directly yields topologically correct, three-di…
Whole brain segmentation and cortical surface parcellation are essential in understanding the anatomical-functional relationships of the brain. Multi-atlas segmentation has been regarded as one of the leading segmentation methods for the whole brain segmentation. In our recent work, the multi-atlas technique has been adapted to surface reconstruction using a method called Multi-atlas CRUISE (MaCRUISE). The MaCRUISE method not only performed consistent volume-surface analyses but also showed advantages on robustness compared with the FreeSurfer method. However, a detailed surface parcellation w…
Shared representations in brains and models reveal a two-route cortical organization during scene perception
Department of Cognition, Development and Education Psychology, Faculty of Psychology, University of Barcelona（认知、发展与教育心理学系，心理学学院，巴塞罗那大学）；Institute of Neurosciences, University of Barcelona（神经科学研究所，巴塞罗那大学）；Bellvitge Institute for Biomedical Research（Bellvitge生物医学研究 institute）
University of North Carolina at Chapel Hill（北卡罗来纳大学教堂山分校）
Charting cortical growth trajectories is of paramount importance for understanding brain development. However, such analysis necessitates the collection of longitudinal data, which can be challenging due to subject dropouts and failed scans. In this paper, we will introduce a method for longitudinal prediction of cortical surfaces using a spatial graph convolutional neural network (GCNN), which extends conventional CNNs from Euclidean to curved manifolds. The proposed method is designed to model the cortical growth trajectories and jointly predict inner and outer cortical surfaces at multiple …
Massachusetts Institute of Technology（麻省理工学院）；Free University Berlin（柏林自由大学）
The complex multi-stage architecture of cortical visual pathways provides the neural basis for efficient visual object recognition in humans. However, the stage-wise computations therein remain poorly understood. Here, we compared temporal (magnetoencephalography) and spatial (functional MRI) visual brain representations with representations in an artificial deep neural network (DNN) tuned to the statistics of real-world visual recognition. We showed that the DNN captured the stages of human visual processing in both time and space from early visual areas towards the dorsal and ventral streams…
Learning Cortical Anomaly through Masked Encoding for Unsupervised Heterogeneity Mapping
Department of Psychiatry and Psychotherapy, Tübingen Center for Mental Health (TÜCMH)（图宾根心理健康中心精神病学与心理治疗系）；Norwegian Centre for Mental Disorders Research（挪威精神障碍研究中心）；Donders Institute for Brain, Cognition, and Behaviour, Radboud University（拉德堡德大学唐德斯脑、认知与行为研究所）
University of Queensland（昆士兰大学）；Queensland Brain Institute（昆士兰脑研究所）
The emergence of explainability methods has enabled a better comprehension of how deep neural networks operate through concepts that are easily understood and implemented by the end user. While most explainability methods have been designed for traditional deep learning, some have been further developed for geometric deep learning, in which data are predominantly represented as graphs. These representations are regularly derived from medical imaging data, particularly in the field of neuroimaging, in which graphs are used to represent brain structural and functional wiring patterns (brain conn…
University of Pennsylvania（宾夕法尼亚大学）；Lund University（隆德大学）
Ex vivo MRI of the brain provides remarkable advantages over in vivo MRI for visualizing and characterizing detailed neuroanatomy. However, automated cortical segmentation methods in ex vivo MRI are not well developed, primarily due to limited availability of labeled datasets, and heterogeneity in scanner hardware and acquisition protocols. In this work, we present a high resolution 7 Tesla dataset of 32 ex vivo human brain specimens. We benchmark the cortical mantle segmentation performance of nine neural network architectures, trained and evaluated using manually-segmented 3D patches sampled…
Recon-all-clinical: Cortical surface reconstruction and analysis of heterogeneous clinical brain MRI
Massachusetts General Hospital（麻省总医院）；Harvard Medical School（哈佛医学院）；Copenhagen University Hospital - Amager and Hvidovre（哥本哈根大学医院阿莫尔与维杜夫院区）；Danish Research Centre for Magnetic Resonance（丹麦磁共振研究中心）；Centre for Functional and Diagnostic Imaging and Research（功能与诊断成像及研究中心）
We present a hybrid method that performs the complete parcellation of the cerebral cortex of an individual, based on the connectivity information of the white matter fibers from a whole-brain tractography dataset. The method consists of five steps, first intra-subject clustering is performed on the brain tractography. The fibers that make up each cluster are then intersected with the cortical mesh and then filtered to discard outliers. In addition, the method resolves the overlapping between the different intersection regions (sub-parcels) throughout the cortex efficiently. Finally, a post-pro…
International Institute of Information Technology (IIIT)（国际信息技术学院）；McGill University（麦吉尔大学）；Sree Chitra Tirunal Institute for Medical Sciences and Technology（Sree Chitra Tirunal医学科学与技术研究所）
Segmentation of sub-cortical structures from MRI scans is of interest in many neurological diagnosis. Since this is a laborious task machine learning and specifically deep learning (DL) methods have become explored. The structural complexity of the brain demands a large, high quality segmentation dataset to develop good DL-based solutions for sub-cortical structure segmentation. Towards this, we are releasing a set of 114, 1.5 Tesla, T1 MRI scans with manual delineations for 14 sub-cortical structures. The scans in the dataset were acquired from healthy young (21-30 years) subjects ( 58 male a…
Imperial College London（帝国理工学院）；King’s College London（伦敦国王学院）；Centre for the Developing Brain（发育中的大脑中心）；School of Biomedical Engineering and Imaging Sciences（生物医学工程与成像科学学院）；Dept. Computing, BioMedIA（计算机系，生物医学信息学（BioMedIA））；ISBE（（伦敦国王学院）生物医学工程与成像科学学院（School of Biomedical Engineering and Imaging Sciences））
The cerebral cortex performs higher-order brain functions and is thus implicated in a range of cognitive disorders. Current analysis of cortical variation is typically performed by fitting surface mesh models to inner and outer cortical boundaries and investigating metrics such as surface area and cortical curvature or thickness. These, however, take a long time to run, and are sensitive to motion and image and surface resolution, which can prohibit their use in clinical settings. In this paper, we instead propose a machine learning solution, training a novel architecture to predict cortical t…
Deep convolutional neural networks (CNNs) trained on objects and scenes have shown intriguing ability to predict some response properties of visual cortical neurons. However, the factors and computations that give rise to such ability, and the role of intermediate processing stages in explaining changes that develop across areas of the cortical hierarchy, are poorly understood. We focused on the sensitivity to textures as a paradigmatic example, since recent neurophysiology experiments provide rich data pointing to texture sensitivity in secondary but not primary visual cortex. We developed a …
The varying cortical geometry of the brain creates numerous challenges for its analysis. Recent developments have enabled learning surface data directly across multiple brain surfaces via graph convolutions on cortical data. However, current graph learning algorithms do fail when brain surface data are misaligned across subjects, thereby affecting their ability to deal with data from multiple domains. Adversarial training is widely used for domain adaptation to improve the segmentation performance across domains. In this paper, adversarial training is exploited to learn surface data across inc…
CSIRO Health & Biosecurity, The Australian eHealth Research Centre（澳大利亚联邦科学与工业研究组织（CSIRO）健康与生物安全部，澳大利亚电子健康研究中心）；Queensland University of Technology (QUT)（昆士兰科技大学（QUT））；Image and Video Laboratory（图像与视频实验室）；CSIRO Data61（澳大利亚联邦科学与工业研究组织（CSIRO）数据61部（Data61））；CSIRO Data61, Brisbane, Australia（澳大利亚联邦科学与工业研究组织（CSIRO）数据61部，布里斯班，澳大利亚）；The Australian eHealth Research Centre（澳大利亚电子健康研究中心）；Brisbane, QLD, Australia（布里斯班，昆士兰州，澳大利亚）；Brisbane, Australia（布里斯班，澳大利亚）
The study of neurodegenerative diseases relies on the reconstruction and analysis of the brain cortex from magnetic resonance imaging (MRI). Traditional frameworks for this task like FreeSurfer demand lengthy runtimes, while its accelerated variant FastSurfer still relies on a voxel-wise segmentation which is limited by its resolution to capture narrow continuous objects as cortical surfaces. Having these limitations in mind, we propose DeepCSR, a 3D deep learning framework for cortical surface reconstruction from MRI. Towards this end, we train a neural network model with hypercolumn features…
University of Wolverhampton（伍尔弗汉普顿大学）；São Paulo State University（圣保罗州立大学）；Edinburgh Napier University（爱丁堡龙比亚大学）
Despite the recent success of machine learning algorithms, most models face drawbacks when considering more complex tasks requiring interaction between different sources, such as multimodal input data and logical time sequences. On the other hand, the biological brain is highly sharpened in this sense, empowered to automatically manage and integrate such streams of information. In this context, this work draws inspiration from recent discoveries in brain cortical circuits to propose a more biologically plausible self-supervised machine learning approach. This combines multimodal information us…
Institute of Neuroscience and Medicine (INM-1), Research Centre Jülich（于利希研究中心神经科学与医学研究所（INM-1））；Helmholtz AI, Research Centre Jülich（于利希研究中心亥姆霍兹人工智能）；Heinrich Heine University, Düsseldorf（杜塞尔多夫海因里希·海涅大学）；Cécile & Oscar Vogt Institute for Brain Research, University Hospital Düsseldorf（杜塞尔多夫大学医院塞西尔和奥斯卡·沃格特脑研究所）
Cytoarchitecture describes the spatial organization of neuronal cells in the brain, including their arrangement into layers and columns with respect to cell density, orientation, or presence of certain cell types. It allows to segregate the brain into cortical areas and subcortical nuclei, links structure with connectivity and function, and provides a microstructural reference for human brain atlases. Mapping boundaries between areas requires to scan histological sections at microscopic resolution. While recent high-throughput scanners allow to scan a complete human brain in the order of a yea…
GOUHFI 2.0: A Next-Generation Toolbox for Brain Segmentation and Cortex Parcellation at Ultra-High Field MRI
Department of Physics, Norwegian University of Science and Technology（挪威科学技术大学物理系）；Department of Radiology and Nuclear Medicine, St. Olavs Hospital HF（斯托尔奥斯医院HF放射科和核医学部）；Department of Neurology and Clinical Neurophysiology, St. Olavs Hospital HF（斯托尔奥斯医院HF神经科和临床神经生理学部）；Department of Neuromedicine and Movement Science, St. Olavs Hospital HF（斯托尔奥斯医院HF神经医学与运动科学部）；Neuro-SysMed Center for Clinical Treatment Research, Department of Neurology, Haukeland University Hospital（Haukeland大学医院神经科临床治疗研究神经系医学中心）；Department of Clinical Medicine, University of Bergen（卑尔根大学临床医学系）；K.G. Jebsen Center for Translational Research in Parkinson’s disease, University of Bergen（帕金森病转化研究Jebsen中心，卑尔根大学）
AtlasSeg: Atlas Prior Guided Dual-U-Net for Cortical Segmentation in Fetal Brain MRI
College of Biomedical Engineering & Instrument Science, Zhejiang University（浙江大学生物医学工程与仪器科学学院）；Beijing Hospital, National Center of Gerontology, Institute of Geriatric Medicine, Chinese Academy of Medical Sciences（中国医学科学院北京医院国家老年医学中心老年医学研究所）；Shandong Provincial Hospital Affiliated to Shandong First Medical University（山东第一医科大学附属山东省立医院）；State Key Lab of CAD&CG, Zhejiang University（浙江大学CAD&CG国家重点实验室）
The neonatal cortical surface is known to be affected by preterm birth, and the subsequent changes to cortical organisation have been associated with poorer neurodevelopmental outcomes. Deep Generative models have the potential to lead to clinically interpretable models of disease, but developing these on the cortical surface is challenging since established techniques for learning convolutional filters are inappropriate on non-flat topologies. To close this gap, we implement a surface-based CycleGAN using mixture model CNNs (MoNet) to translate sphericalised neonatal cortical surface features…
