Functional magnetic resonance imaging (fMRI) techniques have contributed significantly to our understanding of brain function. Current methods are based on the analysis of \emph{gradual and continuous} changes in the brain blood oxygenated level dependent (BOLD) signal. Departing from that approach, recent work has shown that equivalent results can be obtained by inspecting only the relatively large amplitude BOLD signal peaks, suggesting that relevant information can be condensed in \emph{discrete} events. This idea is further explored here to demonstrate how brain dynamics at resting state can be captured just by the timing and location of such events, i.e., in terms of a spatiotemporal point process. As a proof of principle, we show that the resting state networks (RSN) maps can be extracted from such point processes. Furthermore, the analysis uncovers avalanches of activity which are ruled by the same dynamical and statistical properties described previously for neuronal events at smaller scales. Given the demonstrated functional relevance of the resting state brain dynamics, its representation as a discrete process might facilitate large scale analysis of brain function both in health and disease.
BrainWorld: A Structural-Prior-Conditioned Generative Model for Whole-Brain 4D fMRI Dynamics
Department of Biomedical Engineering, Southern University of Science and Technology（南方科技大学生物医学工程系）；School of Biomedical Engineering, Shenzhen University（深圳大学生物医学工程学院）
Brain atlases are a ubiquitous tool used for analyzing and interpreting brain imaging datasets. Traditionally, brain atlases divided the brain into regions separated by anatomical landmarks. In the last decade, several attempts have been made to parcellate the brain into regions with distinct functional activity using fMRI. To construct a brain atlas using fMRI, data driven algorithms are used to group voxels with similar functional activity together to form regions. Hierarchical clustering is one parcellation method that has been used for functional parcellation of the brain, resulting in par…
In this paper we review the preprocessing pipeline through which fMRI data is transformed into a network. We discuss three parameters that mostly affect our understanding of the existence of functional correlations between the brain regions. In the end, we conclude that the existence of functional correlations between pairs of the brain's regions can be modeled with probabilistic edges, not to lose the uncertainty that is inherent in the network generation process.
Brain-mapping techniques have proven to be vital in understanding the molecular, cellular, and functional mechanisms of the brain. Normal anatomical imaging can provide structural information on certain abnormalities in the brain. However there are many neurological disorders for which only structure studies are not sufficient. In such cases it is required to investigate the functional organization of the brain. Further it is necessary to study the brain functions under normal as well as diseased conditions. Brain mapping techniques can help in deriving useful and important information on thes…
Recent studies have shown that multi-modeling methods can provide new insights into the analysis of brain components that are not possible when each modality is acquired separately. The joint representations of different modalities is a robust model to analyze simultaneously acquired electroencephalography and functional magnetic resonance imaging (EEG-fMRI). Advances in precision instruments have given us the ability to observe the spatiotemporal neural dynamics of the human brain through non-invasive neuroimaging techniques such as EEG & fMRI. Nonlinear fusion methods of streams can extract …
It has become increasingly popular to study the brain as a network due to the realization that functionality cannot be explained exclusively by independent activation of specialized regions. Instead, across a large spectrum of behaviors, function arises due to the dynamic interactions between brain regions. The existing literature on functional brain networks focuses mainly on a battery of network properties characterizing the "resting state" using for example the modularity, clustering, or path length among regions. In contrast, we seek to uncover subgraphs of functional connectivity that pre…
Human memory -- the learning of new information involves changes at the synaptic level between neurons dedicated for storage of in-formation. Generally, memory is classified as Long-Term Memory and Short-Term Memory. The various types of the memory and their disorder are widely studied using neuroimaging techniques like Electroencephalography (EEG) and functional Magnetic Resonance Imaging (fMRI). Brain is effectively occupied with the capabilities of learning, retention and recall. The brain regions (pre-frontal cortex, associated hippocampus cortices and their interactions with other lobes) …
Functional magnetic resonance imaging (fMRI) is one of the most common imaging modalities to investigate brain functions. Recent studies in neuroscience stress the great potential of functional brain networks constructed from fMRI data for clinical predictions. Traditional functional brain networks, however, are noisy and unaware of downstream prediction tasks, while also incompatible with the deep graph neural network (GNN) models. In order to fully unleash the power of GNNs in network-based fMRI analysis, we develop FBNETGEN, a task-aware and interpretable fMRI analysis framework via deep br…
Brain decoding, aiming to identify the brain states using neural activity, is important for cognitive neuroscience and neural engineering. However, existing machine learning methods for fMRI-based brain decoding either suffer from low classification performance or poor explainability. Here, we address this issue by proposing a biologically inspired architecture, Spatial Temporal-pyramid Graph Convolutional Network (STpGCN), to capture the spatial-temporal graph representation of functional brain activities. By designing multi-scale spatial-temporal pathways and bottom-up pathways that mimic th…
The formation of transient networks in response to external stimuli or as a reflection of internal cognitive processes is a hallmark of human brain function. However, its identification in fMRI data of the human brain is notoriously difficult. Here we propose a new method of fMRI data analysis that tackles this problem by considering large-scale, task-related synchronisation networks. Networks consist of nodes and edges connecting them, where nodes correspond to voxels in fMRI data, and the weight of an edge is determined via task-related changes in dynamic synchronisation between their respec…
Georgia Institute of Technology（佐治亚理工学院）；Emory University（埃默里大学）；Carnegie Mellon University（卡内基梅隆大学）；University of California, Los Angeles（加州大学洛杉矶分校）
Functional magnetic resonance imaging (fMRI) has become one of the most common imaging modalities for brain function analysis. Recently, graph neural networks (GNN) have been adopted for fMRI analysis with superior performance. Unfortunately, traditional functional brain networks are mainly constructed based on similarities among region of interests (ROI), which are noisy and agnostic to the downstream prediction tasks and can lead to inferior results for GNN-based models. To better adapt GNNs for fMRI analysis, we propose TBDS, an end-to-end framework based on \underline{T}ask-aware \underlin…
Large Language Model-based FMRI Encoding of Language Functions for Subjects with Neurocognitive Disorder
The Chinese University of Hong Kong（香港中文大学）；Stanley Ho Big Data Decision Analytics Research Centre, The Chinese University of Hong Kong（香港中文大学何文汇大数据决策分析研究中心）
Estimated connectomes by the means of neuroimaging techniques have enriched our knowledge of the organizational properties of the brain leading to the development of network-based clinical diagnostics. Unfortunately, to date, many of those network-based clinical diagnostics tools, based on the mere description of isolated instances of observed connectomes are noisy estimates of the true connectivity network. Modeling brain connectivity networks is therefore important to better explain the functional organization of the brain and allow inference of specific brain properties. In this report, we …
National Kapodistrian Univ. of Athens (NKUA/UoA)（雅典国家卡波季斯特里安大学）
Functional Magnetic Resonance Imaging (fMRI) is a powerful non-invasive tool for localizing and analyzing brain activity. This study focuses on one very important aspect of the functional properties of human brain, specifically the estimation of the level of parallelism when performing complex cognitive tasks. Using fMRI as the main modality, the human brain activity is investigated through a purely data-driven signal processing and dimensionality analysis approach. Specifically, the fMRI signal is treated as a multi-dimensional data space and its intrinsic `complexity' is studied via dataset …
Brain-DiT: A Universal Multi-state fMRI Foundation Model with Metadata-Conditioned Pretraining
Department of Biomedical Engineering, Southern University of Science and Technology, China（南方科技大学生物医学工程系，中国）；School of Biomedical Engineering, Shenzhen University, China（深圳大学生物医学工程学院，中国）
Brain-MGF: Multimodal Graph Fusion Network for EEG-fMRI Brain Connectivity Analysis Under Psilocybin
School of Information Technology, Monash University Malaysia（信息技术学院，墨尔本大学马来西亚分校）；Department of Data Science and AI, Faculty of Information Technology, Monash University（数据科学与人工智能系，信息科技学院，墨尔本大学）；Turner Institute for Brain and Mental Health, School of Psychological Sciences, Monash University（脑与心理健康研究所，心理学科学学院，墨尔本大学）
HOI-Brain: a novel multi-channel transformers framework for brain disorder diagnosis by accurately extracting signed higher-order interactions from fMRI
The functional network approach, where fMRI BOLD time series are mapped to networks depicting functional relationships between brain areas, has opened new insights into the function of the human brain. In this approach, the choice of network nodes is of crucial importance. One option is to consider fMRI voxels as nodes. This results in a large number of nodes, making network analysis and interpretation of results challenging. A common alternative is to use pre-defined clusters of anatomically close voxels, Regions of Interest (ROIs). This approach assumes that voxels within ROIs are functional…
In the past few years, significant advancements were made in reconstruction of observed natural images from fMRI brain recordings using deep-learning tools. Here, for the first time, we show that dense 3D depth maps of observed 2D natural images can also be recovered directly from fMRI brain recordings. We use an off-the-shelf method to estimate the unknown depth maps of natural images. This is applied to both: (i) the small number of images presented to subjects in an fMRI scanner (images for which we have fMRI recordings - referred to as "paired" data), and (ii) a very large number of natura…
A great improvement to the insight on brain function that we can get from fMRI data can come from effective connectivity analysis, in which the flow of information between even remote brain regions is inferred by the parameters of a predictive dynamical model. As opposed to biologically inspired models, some techniques as Granger causality (GC) are purely data-driven and rely on statistical prediction and temporal precedence. While powerful and widely applicable, this approach could suffer from two main limitations when applied to BOLD fMRI data: confounding effect of hemodynamic response func…
Interpretable Information-Decomposed Brain Graph Learning for fMRI-based Disease Diagnosis
School of Mathematics and Statistics, Shandong University（山东大学数学与统计学院）；Data Science Institute, Shandong University（山东大学数据科学研究院）；Academy of Mathematics and Systems Science, University of Chinese Academy of Sciences（中国科学院大学数学与系统科学研究院）
University of Rome, Tor Vergata（罗马第二大学）；Istituto di Bioimmagini e Fisiologia Molecolare, CNR（意大利国家研究委员会生物成像与分子生理学研究所）；Martinos Center for Biomedical Imaging（马丁诺斯生物医学成像中心）；MGH（麻省总医院）；Harvard Medical School（哈佛医学院）
Brain decoding is a field of computational neuroscience that uses measurable brain activity to infer mental states or internal representations of perceptual inputs. Therefore, we propose a novel approach to brain decoding that also relies on semantic and contextual similarity. We employ an fMRI dataset of natural image vision and create a deep learning decoding pipeline inspired by the existence of both bottom-up and top-down processes in human vision. We train a linear brain-to-feature model to map fMRI activity features to visual stimuli features, assuming that the brain projects visual info…
Fusion Learning from Dynamic Functional Connectivity: Combining the Amplitude and Phase of fMRI Signals to Identify Brain Disorders
Guangdong Provincial Key Laboratory of Multimodal Big Data Intelligent Analysis（广东省多模态大数据智能分析重点实验室）；School of Computer Science and Engineering, South China University of Technology（华南理工大学计算机科学与工程学院）
Neurofeedback is a non-invasive brain training with long-term medical and non-medical applications. Despite the existence of several emotion regulation studies using neurofeedback, further investigation is needed to understand interactions of the brain regions involved in the process. We implemented EEG neurofeedback with simultaneous fMRI using a modified happiness-inducing task through autobiographical memories to upregulate positive emotion. The results showed increased activity of prefrontal, occipital, parietal, and limbic regions and increased functional connectivity between prefrontal, …
Ensemble-based graph representation of fMRI data for cognitive brain state classification
Institute for Cognitive Neuroscience, University Higher School of Economics（认知神经科学研究所，高等经济大学）；Department of Mathematics and Institute for Women’s Health, University College London（数学系和妇女健康研究所，伦敦大学学院）
Anatomy of the human brain constrains the formation of large-scale functional networks. Here, given measured brain activity in gray matter, we interpolate these functional signals into the white matter on a structurally-informed high-resolution voxel-level brain grid. The interpolated volumes reflect the underlying anatomical information, revealing white matter structures that mediate functional signal flow between temporally coherent gray matter regions. Functional connectivity analyses of the interpolated volumes reveal an enriched picture of the default mode network (DMN) and its subcompone…
Correlations in the signal observed via functional Magnetic Resonance Imaging (fMRI), are expected to reveal the interactions in the underlying neural populations through hemodynamic response. In particular, they highlight distributed set of mutually correlated regions that correspond to brain networks related to different cognitive functions. Yet graph-theoretical studies of neural connections give a different picture: that of a highly integrated system with small-world properties: local clustering but with short pathways across the complete structure. We examine the conditional independence …
Southwest University（西南大学）；Collaborative Innovation Center of Assessment Toward Basic Education Quality at Beijing Normal University（北京师范大学基础教育质量评价协同创新中心）
Major depressive disorder (MDD) is one of the most common mental health conditions that has been intensively investigated for its association with brain atrophy and mortality. Recent studies reveal that the deviation between the predicted and the chronological age can be a marker of accelerated brain aging to characterize MDD. However, current conclusions are usually drawn based on structural MRI information collected from Caucasian participants. The universality of this biomarker needs to be further validated by subjects with different ethnic/racial backgrounds and by different types of data.…
Functional Magnetic Resonance Images acquired during resting-state provide information about the functional organization of the brain through measuring correlations between brain areas. Independent components analysis is the reference approach to estimate spatial components from weakly structured data such as brain signal time courses; each of these components may be referred to as a brain network and the whole set of components can be conceptualized as a brain functional atlas. Recently, new methods using a sparsity prior have emerged to deal with low signal-to-noise ratio data. However, even…
Yale University（耶鲁大学）；Yale School of Medicine（耶鲁医学院）
Autism spectrum disorder (ASD) is a complex neurodevelopmental disorder. Finding the biomarkers associated with ASD is extremely helpful to understand the underlying roots of the disorder and can lead to earlier diagnosis and more targeted treatment. Although Deep Neural Networks (DNNs) have been applied in functional magnetic resonance imaging (fMRI) to identify ASD, understanding the data-driven computational decision making procedure has not been previously explored. Therefore, in this work, we address the problem of interpreting reliable biomarkers associated with identifying ASD; specific…
University of Brescia（布雷西亚大学）；Tel Aviv Sourasky Medical Center（特拉维夫苏拉斯基医疗中心）；Maastricht University（马斯特里赫特大学）
Deep neural networks have been developed drawing inspiration from the brain visual pathway, implementing an end-to-end approach: from image data to video object classes. However building an fMRI decoder with the typical structure of Convolutional Neural Network (CNN), i.e. learning multiple level of representations, seems impractical due to lack of brain data. As a possible solution, this work presents the first hybrid fMRI and deep features decoding approach: collected fMRI and deep learnt representations of video object classes are linked together by means of Kernel Canonical Correlation Ana…
Functional magnetic resonance imaging (fMRI) is widely used in clinical applications to highlight brain areas involved in specific cognitive processes. Brain impairments, such as tumors, suppress the fMRI activation of the anatomical areas they invade and, thus, brain-damaged functional networks present missing links/areas of activation. The identification of the missing circuitry components is of crucial importance to estimate the damage extent. The study of functional networks associated to clinical tasks but performed by healthy individuals becomes, therefore, of paramount concern. These `h…
Recently, we reported an emotion self-regulation study (Zotev et al., 2020), in which patients with major depressive disorder (MDD) used simultaneous real-time fMRI and EEG neurofeedback (rtfMRI-EEG-nf) to upregulate two fMRI and two EEG activity measures, relevant to MDD. The target measures included fMRI activities of the left amygdala and left rostral anterior cingulate cortex, and frontal EEG asymmetries in the alpha band (FAA) and high-beta band (FBA). Here we apply the exact low resolution brain electromagnetic tomography (eLORETA) to investigate EEG source activities during the rtfMRI-E…
Neurofeedback is a promising approach for non-invasive modulation of human brain activity with applications for treatment of mental disorders and enhancement of brain performance. Neurofeedback techniques are commonly based on either electroencephalography (EEG) or real-time functional magnetic resonance imaging (rtfMRI). Advances in simultaneous EEG-fMRI have made it possible to combine the two approaches. Here we report the first implementation of simultaneous multimodal rtfMRI and EEG neurofeedback (rtfMRI-EEG-nf). It is based on a novel system for real-time integration of simultaneous rtfM…
The study of hierarchy in networks of the human brain has been of significant interest among the researchers as numerous studies have pointed out towards a functional hierarchical organization of the human brain. This paper provides a novel method for the extraction of hierarchical connectivity components in the human brain using resting-state fMRI. The method builds upon prior work of Sparse Connectivity Patterns (SCPs) by introducing a hierarchy of sparse overlapping patterns. The components are estimated by deep factorization of correlation matrices generated from fMRI. The goal of the pape…
EmoMind: Decoding Affective Captions from Human Brain fMRI
Department of Biomedical Engineering（生物医学工程系）；Vanderbilt University（范德比大学）；Research Institute of Electrical Communication（电气通信研究所）；Tohoku University（东北大学）；University of Florida（佛罗里达大学）
Tokyo Institute of Technology（东京工业大学）；National Institute of Advanced Industrial Science and Technology（产业技术综合研究所）；AIST-Tokyo Tech Real World Big-Data Computation Open Innovation Laboratory（AIST-东京工业大学真实世界大数据计算开放创新实验室）
With the advent of brain imaging techniques and machine learning tools, much effort has been devoted to building computational models to capture the encoding of visual information in the human brain. One of the most challenging brain decoding tasks is the accurate reconstruction of the perceived natural images from brain activities measured by functional magnetic resonance imaging (fMRI). In this work, we survey the most recent deep learning methods for natural image reconstruction from fMRI. We examine these methods in terms of architectural design, benchmark datasets, and evaluation metrics …
