Enhancing Angular Resolution via Directionality Encoding and Geometric Constraints in Brain Diffusion Tensor Imaging
School of Computer Science, University of Sydney（悉尼大学计算机科学学院）；Brain and Mind Centre, University of Sydney（悉尼大学脑与心智中心）；Central Clinical School, University of Sydney（悉尼大学临床学院）；Sydney Neuroimaging Analysis Centre, University of Sydney（悉尼大学悉尼神经影像分析中心）；School of Biomedical Engineering, University of Sydney（悉尼大学生物医学工程学院）；Sydney Imaging, University of Sydney（悉尼大学悉尼影像中心）
Sheng Chen, Zihao Tang, Mariano Cabezas, Xinyi Wang, Arkiev D'Souza, Michael Barnett, Fernando Calamante, Weidong Cai, Chenyu Wang
Diffusion-weighted imaging (DWI) is a type of Magnetic Resonance Imaging (MRI) technique sensitised to the diffusivity of water molecules, offering the capability to inspect tissue microstructures and is the only in-vivo method to reconstruct white matter fiber tracts non-invasively. The DWI signal can be analysed with the diffusion tensor imaging (DTI) model to estimate the directionality of water diffusion within voxels. Several scalar metrics, including axial diffusivity (AD), mean diffusivity (MD), radial diffusivity (RD), and fractional anisotropy (FA), can be further derived from DTI to quantitatively summarise the microstructural integrity of brain tissue. These scalar metrics have played an important role in understanding the organisation and health of brain tissue at a microscopic level in clinical studies. However, reliable DTI metrics rely on DWI acquisitions with high gradient directions, which often go beyond the commonly used clinical protocols. To enhance the utility of clinically acquired DWI and save scanning time for robust DTI analysis, this work proposes DirGeo-DTI, a deep learning-based method to estimate reliable DTI metrics even from a set of DWIs acquired with the minimum theoretical number (6) of gradient directions. DirGeo-DTI leverages directional encoding and geometric constraints to facilitate the training process. Two public DWI datasets were used for evaluation, demonstrating the effectiveness of the proposed method. Extensive experimental results show that the proposed method achieves the best performance compared to existing DTI enhancement methods and potentially reveals further clinical insights with routine clinical DWI scans.
Scaling up ridge regression for brain encoding in a massive individual fMRI dataset
Concordia University（康考迪亚大学）；Université de Montréal（蒙特利尔大学）；Centre de Recherche de L’Institut Universitaire de Gériatrie de Montréal（蒙特利尔大学老年医学研究所研究中心）
University at Buffalo, The State University of New York（纽约州立大学布法罗分校）
Brain Tumor Segmentation from magnetic resonance imaging (MRI) is a critical technique for early diagnosis. However, rather than having complete four modalities as in BraTS dataset, it is common to have missing modalities in clinical scenarios. We design a brain tumor segmentation algorithm that is robust to the absence of any modality. Our network includes a channel-independent encoding path and a feature-fusion decoding path. We use self-supervised training through channel dropout and also propose a novel domain adaptation method on feature maps to recover the information from the missing ch…
Princeton University（普林斯顿大学）；Max Planck Institute for Software Systems（马克斯·普朗克软件系统研究所）；Carnegie Mellon University（卡内基梅隆大学）；Google Research（谷歌研究院）
To study information processing in the brain, neuroscientists manipulate experimental stimuli while recording participant brain activity. They can then use encoding models to find out which brain "zone" (e.g. which region of interest, volume pixel or electrophysiology sensor) is predicted from the stimulus properties. Given the assumptions underlying this setup, when stimulus properties are predictive of the activity in a zone, these properties are understood to cause activity in that zone. In recent years, researchers have used neural networks to construct representations that capture the div…
Enabling effective brain-computer interfaces requires understanding how the human brain encodes stimuli across modalities such as visual, language (or text), etc. Brain encoding aims at constructing fMRI brain activity given a stimulus. There exists a plethora of neural encoding models which study brain encoding for single mode stimuli: visual (pretrained CNNs) or text (pretrained language models). Few recent papers have also obtained separate visual and text representation models and performed late-fusion using simple heuristics. However, previous work has failed to explore: (a) the effective…
DCL-SE: Dynamic Curriculum Learning for Spatiotemporal Encoding of Brain Imaging
University of Chinese Academy of Sciences（中国科学院大学）；Capital Medical University（首都医科大学）；Beijing Tongren Hospital（北京同仁医院）；Beijing Institute of Ophthalmology（北京眼科研究所）；Wannan Medical University（皖南医学院）
Sahand University of Technology（萨汉德工业大学）；Lund University（隆德大学）
The human brain cortical layer has a convoluted morphology that is unique to each individual. Characterization of the cortical morphology is necessary in longitudinal studies of structural brain change, as well as in discriminating individuals in health and disease. A method for encoding the cortical morphology in the form of a graph is presented. The design of graphs that encode the global cerebral hemisphere cortices as well as localized cortical regions is proposed. Spectral features of these graphs are then studied and proposed as descriptors of cortical morphology. As proof-of-concept of …
Segmentation of brain structures from magnetic resonance (MR) scans plays an important role in the quantification of brain morphology. Since 3D deep learning models suffer from high computational cost, 2D deep learning methods are favored for their computational efficiency. However, existing 2D deep learning methods are not equipped to effectively capture 3D spatial contextual information that is needed to achieve accurate brain structure segmentation. In order to overcome this limitation, we develop an Anatomical Context-Encoding Network (ACEnet) to incorporate 3D spatial and anatomical conte…
Training Compute-Optimal Vision Transformers for Brain Encoding
Concordia University（康考迪亚大学）；Université de Montréal（蒙特利尔大学）；Centre de Recherche de L’Institut Universitaire de Gériatrie de Montréal（蒙特利尔大学老年医学研究所研究中心）
ViBE: Visual-to-M/EEG Brain Encoding via Spatio-Temporal VAE and Distribution-Aligned Projection
Jinan University（济南大学）；The First Affiliated Hospital of Jinan University（济南大学第一附属医院）；Western University（西方大学）；Guangdong University of Technology（广东工业大学）；Tsinghua University（清华大学）
Probing Multimodal Fusion in the Brain: The Dominance of Audiovisual Streams in Naturalistic Encoding
Distributed and Intelligent Optimization Research Laboratory（分布式智能优化研究实验室）；Electrical Engineering Department（电气工程系）；Amirkabir University of Technology（阿米尔卡比尔技术大学）
Deep Neural Networks and Brain Alignment: Brain Encoding and Decoding (Survey)
Inria Bordeaux（法国国家信息与自动化研究所波尔多分所）；LaBRI, Université de Bordeaux（波尔多大学拉布里实验室）；Université de Bordeaux（波尔多大学）；TU Berlin（柏林工业大学）；National University of Singapore（新加坡国立大学）；Microsoft（微软公司）；International Institute of Information Technology Hyderabad（海得拉巴国际信息技术研究所）；CEA（法国原子能和替代能源委员会）；CNRS（法国国家科学研究中心）
The mechanisms involved in transforming early visual signals to curvature representations in V4 are unknown. We propose a hierarchical model that reveals V1/V2 encodings that are essential components for this transformation to the reported curvature representations in V4. Then, by relaxing the often-imposed prior of a single Gaussian, V4 shape selectivity is learned in the last layer of the hierarchy from Macaque V4 responses. We found that V4 cells integrate multiple shape parts from the full spatial extent of their receptive fields with similar excitatory and inhibitory contributions. Our re…
National Digital Switching System Engineering and Technological Research Center（国家数字交换系统工程技术研究中心）；University of Minnesota at Twin Cities（明尼苏达大学双城分校）
Background: Building visual encoding models to accurately predict visual responses is a central challenge for current vision-based brain-machine interface techniques. To achieve high prediction accuracy on neural signals, visual encoding models should include precise visual features and appropriate prediction algorithms. Most existing visual encoding models employ hand-craft visual features (e.g., Gabor wavelets or semantic labels) or data-driven features (e.g., features extracted from deep neural networks (DNN)). They also assume a linear mapping between feature representation to brain activi…
Korea Advanced Institute of Science and Technology（韩国科学技术院）
As deep learning is showing unprecedented success in medical image analysis tasks, the lack of sufficient medical data is emerging as a critical problem. While recent attempts to solve the limited data problem using Generative Adversarial Networks (GAN) have been successful in generating realistic images with diversity, most of them are based on image-to-image translation and thus require extensive datasets from different domains. Here, we propose a novel model that can successfully generate 3D brain MRI data from random vectors by learning the data distribution. Our 3D GAN model solves both i…
Mechanistic Learning with Guided Diffusion Models to Predict Spatio-Temporal Brain Tumor Growth
ETH Zurich, 8092 Zurich, Switzerland - SIB Swiss Institue of Bioinformatics, 1015 Lausanne, Switzerland Department of Radiation Oncology, Inselspital, Bern University Hospital；University of Bern, Switzerland Sheikh Zayed Institute for Pediatric Surgical Innovation, Children’s National Medical Center, Washington, DC 20010, United States University of California San Francisco, San Francisco, CA 94143, United States University Children's Hospital Zurich, 8008 Zurich, Switzerland
Otto von Guericke University Magdeburg（马格德堡奥托·冯·格里克大学）；Center for Behavioral Brain Sciences（行为脑科学研究中心）；German Center for Neurodegenerative Disease（德国神经退行性疾病研究中心）
Expert interpretation of anatomical images of the human brain is the central part of neuro-radiology. Several machine learning-based techniques have been proposed to assist in the analysis process. However, the ML models typically need to be trained to perform a specific task, e.g., brain tumour segmentation or classification. Not only do the corresponding training data require laborious manual annotations, but a wide variety of abnormalities can be present in a human brain MRI - even more than one simultaneously, which renders representation of all possible anomalies very challenging. Hence, …
University of Notre Dame（圣母大学）；Middle East Technical University（中东技术大学）
The main goal of this study is to extract a set of brain networks in multiple time-resolutions to analyze the connectivity patterns among the anatomic regions for a given cognitive task. We suggest a deep architecture which learns the natural groupings of the connectivity patterns of human brain in multiple time-resolutions. The suggested architecture is tested on task data set of Human Connectome Project (HCP) where we extract multi-resolution networks, each of which corresponds to a cognitive task. At the first level of this architecture, we decompose the fMRI signal into multiple sub-bands …
The brain is known to be a highly complex, asynchronous dynamical system that is highly tailored to encode temporal information. However, recent deep learning approaches to not take advantage of this temporal coding. Spiking Neural Networks (SNNs) can be trained using biologically-realistic learning mechanisms, and can have neuronal activation rules that are biologically relevant. This type of network is also structured fundamentally around accepting temporal information through a time-decaying voltage update, a kind of input that current rate-encoding networks have difficulty with. Here we sh…
Middle East Technical University（中东技术大学）；Tohoku University（东北大学）
In this work, we propose a novel framework to encode the local connectivity patterns of brain, using Fisher Vectors (FV), Vector of Locally Aggregated Descriptors (VLAD) and Bag-of-Words (BoW) methods. We first obtain local descriptors, called Mesh Arc Descriptors (MADs) from fMRI data, by forming local meshes around anatomical regions, and estimating their relationship within a neighborhood. Then, we extract a dictionary of relationships, called \textit{brain connectivity dictionary} by fitting a generative Gaussian mixture model (GMM) to a set of MADs, and selecting the codewords at the mean…
Department of Electrical Engineering, Indian Institute of Technology Madras (IITM)（电子工程系，印度理工学院Madras（IITM））；Sudha Gopalakrishnan Brain Centre (SGBC), IITM（Sudha Gopalakrishnan脑中心（SGBC），IITM）
The dispute of how the human brain represents conceptual knowledge has been argued in many scientific fields. Brain imaging studies have shown that the spatial patterns of neural activation in the brain are correlated with thinking about different semantic categories of words (for example, tools, animals, and buildings) or when viewing the related pictures. In this paper, we present a computational model that learns to predict the neural activation captured in functional magnetic resonance imaging (fMRI) data of test words. Unlike the models with hand-crafted features that have been used in th…
Image-to-Brain Signal Generation for Visual Prosthesis with CLIP Guided Multimodal Diffusion Models
Jinan University, Guangzhou, China；Department of Rehabilitation Medicine, The First Affiliated Hospital of Jinan University, Guangzhou, China；Western University, Ontario, Canada；Guangdong University of Technology, Guangzhou, China；Tsinghua University, Beijing, China
Sparse-View Interpretable 3D Animal Behavior Representations for Neural Encoding and Decoding
Harvard University（哈佛大学）；University of Cambridge（剑桥大学）；Northwestern University（西北大学）；Cold Spring Harbor Laboratory（冷泉港实验室）；Columbia University（哥伦比亚大学）；University of Science and Technology of China（中国科学技术大学）
Learning Cortical Anomaly through Masked Encoding for Unsupervised Heterogeneity Mapping
Department of Psychiatry and Psychotherapy, Tübingen Center for Mental Health (TÜCMH)（图宾根心理健康中心精神病学与心理治疗系）；Norwegian Centre for Mental Disorders Research（挪威精神障碍研究中心）；Donders Institute for Brain, Cognition, and Behaviour, Radboud University（拉德堡德大学唐德斯脑、认知与行为研究所）
The human brain cortical layer has a convoluted morphology that is unique to each individual. Characterization of the cortical morphology is necessary in longitudinal studies of structural brain change, as well as in discriminating individuals in health and disease. A method for encoding the cortical morphology in the form of a graph is presented. The design of graphs that encode the global cerebral hemisphere cortices as well as localized cortical regions is proposed. Spectral metrics derived from these graphs are then studied and proposed as descriptors of cortical morphology. As proof-of-co…
