Separating Stimulus-Induced and Background Components of Dynamic Functional Connectivity in Naturalistic fMRI
Monash University Malaysia（莫纳什大学马来西亚分校）；King Abdullah University of Science and Technology（阿卜杜拉国王科技大学）；University College London（伦敦大学学院）；University of Texas at Dallas（德克萨斯大学达拉斯分校）
Chee-Ming Ting, Jeremy I. Skipper, Steven L. Small, Hernando Ombao
备注：
Main paper: 10 pages, 8 figures. Supplemental file: 3 pages
We consider the challenges in extracting stimulus-related neural dynamics from other intrinsic processes and noise in naturalistic functional magnetic resonance imaging (fMRI). Most studies rely on inter-subject correlations (ISC) of low-level regional activity and neglect varying responses in individuals. We propose a novel, data-driven approach based on low-rank plus sparse (L+S) decomposition to isolate stimulus-driven dynamic changes in brain functional connectivity (FC) from the background noise, by exploiting shared network structure among subjects receiving the same naturalistic stimuli. The time-resolved multi-subject FC matrices are modeled as a sum of a low-rank component of correlated FC patterns across subjects, and a sparse component of subject-specific, idiosyncratic background activities. To recover the shared low-rank subspace, we introduce a fused version of principal component pursuit (PCP) by adding a fusion-type penalty on the differences between the rows of the low-rank matrix. The method improves the detection of stimulus-induced group-level homogeneity in the FC profile while capturing inter-subject variability. We develop an efficient algorithm via a linearized alternating direction method of multipliers to solve the fused-PCP. Simulations show accurate recovery by the fused-PCP even when a large fraction of FC edges are severely corrupted. When applied to natural fMRI data, our method reveals FC changes that were time-locked to auditory processing during movie watching, with dynamic engagement of sensorimotor systems for speech-in-noise. It also provides a better mapping to auditory content in the movie than ISC.
Functional neuroimaging can measure the brain?s response to an external stimulus. It is used to perform brain mapping: identifying from these observations the brain regions involved. This problem can be cast into a linear supervised learning task where the neuroimaging data are used as predictors for the stimulus. Brain mapping is then seen as a support recovery problem. On functional MRI (fMRI) data, this problem is particularly challenging as i) the number of samples is small due to limited acquisition time and ii) the variables are strongly correlated. We propose to overcome these difficult…
Sparse linear regression methods including the well-known LASSO and the Dantzig selector have become ubiquitous in the engineering practice, including in medical imaging. Among other tasks, they have been successfully applied for the estimation of neuronal activity from functional magnetic resonance data without prior knowledge of the stimulus or activation timing, utilizing an approximate knowledge of the hemodynamic response to local neuronal activity. These methods work by generating a parametric family of solutions with different sparsity, among which an ultimate choice is made using an in…
Stacked Regression using Off-the-shelf, Stimulus-tuned and Fine-tuned Neural Networks for Predicting fMRI Brain Responses to Movies (Algonauts 2025 Report)
Université Paris Cité（巴黎Cité大学）；University of Oxford（牛津大学）；University of Turin（都灵大学）；Universität Leipzig（莱比锡大学）；Max Planck School of Cognition（马克斯·普朗克认知科学学院）
Naturalistic fMRI has been suggested to be a powerful alternative for investigations of human brain function. Stimulus-induced activation has been playing an essential role in fMRI-based brain function analyses. Due to the complexity of the stimuli, however, detection of activations induced by naturalistic stimuli (AINSs) has been a tricky problem, as AINS cannot be detected simply in a model-driven way. In this study, we proposed a method to detect AINS in a pseudo model-driven way. Inspired by the strategy of utilizing the commonalities among the brains exposed to the same stimuli for inter-…
Huang (arXiv:1612.03270) argues that the perceptual learning induced by our decoded neurofeedback method (DecNef) can be explained by Hebbian synaptic plasticity of connections between V1/V2 and V3/V4 rather than that within V1/V2, and that reinforcement learning at a cellular level should not be possible, and thus challenges our conclusions. In this reply, first, we show that Huang's model is not compatible with our data which strongly suggest plasticity within V1/V2. Second, the results of our new analysis show that spontaneous activity is not random but largely accounted for by low-dimensio…
The shared response model provides a simple but effective framework to analyse fMRI data of subjects exposed to naturalistic stimuli. However when the number of subjects or runs is large, fitting the model requires a large amount of memory and computational power, which limits its use in practice. In this work, we introduce the FastSRM algorithm that relies on an intermediate atlas-based representation. It provides considerable speed-up in time and memory usage, hence it allows easy and fast large-scale analysis of naturalistic-stimulus fMRI data. Using four different datasets, we show that ou…
NeuroWorld: A Latent Brain World Model for Stimulus-Conditioned Human Brain Dynamics
National University of Singapore（新加坡国立大学）；Yong Loo Lin School of Medicine, National University of Singapore（新加坡国立大学杨潞龄医学院）；Department of Medicine, Yong Loo Lin School of Medicine, National University of Singapore（新加坡国立大学杨潞龄医学院医学系）；Department of Electrical and Computer Engineering, National University of Singapore（新加坡国立大学电气与计算机工程系）；Stanford University（斯坦福大学）；Department of Psychology, Stanford University（斯坦福大学心理学系）
In nature, animals encounter high dimensional sensory stimuli that have complex statistical and dynamical structure. Attempts to study the neural coding of these natural signals face challenges both in the selection of the signal ensemble and in the analysis of the resulting neural responses. For zebra finches, naturalistic stimuli can be defined as sounds that they encounter in a colony of conspecific birds. We assembled an ensemble of these sounds by recording groups of 10-40 zebra finches, and then analyzed the response of single neurons in the songbird central auditory area (field L) to co…
A large dataset of human EEG responses to short naturalistic videos for studying dynamic visual event processing
Freie Universität Berlin（柏林自由大学）；Charité – Universitätsmedizin Berlin（柏林夏里特医科大学）；Humboldt-Universität zu Berlin（柏林洪堡大学）；University of Amsterdam（阿姆斯特丹大学）
Brain decoding algorithms form an important part of the arsenal of analysis tools available to neuroscientists, allowing for a more detailed study of the kind of information represented in patterns of cortical activity. While most current decoding algorithms focus on estimating a single, most likely stimulus from the pattern of noisy fMRI responses, the presence of noise causes this estimate to be uncertain. This uncertainty in stimulus estimates is a potentially highly relevant aspect of cortical stimulus processing, and features prominently in Bayesian or probabilistic models of neural codin…
Cornell University（康奈尔大学）；Weill Cornell Medical College（威尔康奈尔医学院）
The increasing popularity of naturalistic paradigms in fMRI (such as movie watching) demands novel strategies for multi-subject data analysis, such as use of neural encoding models. In the present study, we propose a shared convolutional neural encoding method that accounts for individual-level differences. Our method leverages multi-subject data to improve the prediction of subject-specific responses evoked by visual or auditory stimuli. We showcase our approach on high-resolution 7T fMRI data from the Human Connectome Project movie-watching protocol and demonstrate significant improvement ov…
This paper indicates causality as the tool that unifies the analysis of both activations and connectivity of brain areas, obtained with fMRI data. Causality analysis is commonly applied to study connectivity, so this work focuses on demonstrating that also the detection of activations can be handled with a causality analysis. We test our method on finger tapping data, in which GLM and Granger Causality approaches are compared in finding activations. Granger causality not only performs the task well, but indeed we obtained a better localization (i.e. precision) of activations. As a result we cl…
There is a growing interest in joint multi-subject fMRI analysis. The challenge of such analysis comes from inherent anatomical and functional variability across subjects. One approach to resolving this is a shared response factor model. This assumes a shared and time synchronized stimulus across subjects. Such a model can often identify shared information, but it may not be able to pinpoint with high resolution the spatial location of this information. In this work, we examine a searchlight based shared response model to identify shared information in small contiguous regions (searchlights) a…
Institute of Automation, Chinese Academy of Sciences（中国科学院自动化研究所）；Center for Excellence in Brain Science and Intelligence Technology, CAS（中国科学院脑科学与智能技术卓越创新中心）
Decoding human brain activities via functional magnetic resonance imaging (fMRI) has gained increasing attention in recent years. While encouraging results have been reported in brain states classification tasks, reconstructing the details of human visual experience still remains difficult. Two main challenges that hinder the development of effective models are the perplexing fMRI measurement noise and the high dimensionality of limited data instances. Existing methods generally suffer from one or both of these issues and yield dissatisfactory results. In this paper, we tackle this problem by …
Yale University（耶鲁大学）；Université de Montréal（蒙特利尔大学）；Mila – Quebec AI Institute（米拉-魁北克人工智能研究所）
Functional magnetic resonance imaging (fMRI) is a notoriously noisy measurement of brain activity because of the large variations between individuals, signals marred by environmental differences during collection, and spatiotemporal averaging required by the measurement resolution. In addition, the data is extremely high dimensional, with the space of the activity typically having much lower intrinsic dimension. In order to understand the connection between stimuli of interest and brain activity, and analyze differences and commonalities between subjects, it becomes important to learn a meanin…
Princeton University（普林斯顿大学）；Johns Hopkins University（约翰斯·霍普金斯大学）；University of Toronto（多伦多大学）
Several research groups have shown how to correlate fMRI responses to the meanings of presented stimuli. This paper presents new methods for doing so when only a natural language annotation is available as the description of the stimulus. We study fMRI data gathered from subjects watching an episode of BBCs Sherlock [1], and learn bidirectional mappings between fMRI responses and natural language representations. We show how to leverage data from multiple subjects watching the same movie to improve the accuracy of the mappings, allowing us to succeed at a scene classification task with 72% acc…
Independent-Component-Based Encoding Models of Brain Activity During Story Comprehension
School of Electrical and Computer Engineering, Georgia Institute of Technology（佐治亚理工学院电子与计算机工程学院）；School of Psychological and Brain Sciences, Georgia Institute of Technology（佐治亚理工学院心理学与脑科学学院）；Department of Linguistics, Stanford University（斯坦福大学语言学系）
Multimodal Recurrent Ensembles for Predicting Brain Responses to Naturalistic Movies (Algonauts 2025)
Max Planck Institute for Human Cognitive and Brain Sciences（马克斯·普朗克人类认知与脑科学研究所）；TU Dresden（德累斯顿技术大学）；School for Embedded and Composite AI (SECAI)（嵌入式与复合人工智能学院）；Center for Scalable Data Analytics & AI (ScaDS.AI)（可扩展数据与人工智能中心）
School of Computing and Engineering, University of West London（西伦敦大学计算与工程学院）
Objectives: Functional connectivity triggered by naturalistic stimulus (e.g., movies) and machine learning techniques provide a great insight in exploring the brain functions such as fluid intelligence. However, functional connectivity are considered to be multi-layered, while traditional machine learning based on individual models not only are limited in performance, but also fail to extract multi-dimensional and multi-layered information from brain network. Methods: In this study, inspired by multi-layer brain network structure, we propose a new method namely Weighted Ensemble-model and Netw…
We performed simultaneous recordings of electroencephalography (EEG) from multiple students in a classroom, and measured the inter-subject correlation (ISC) of activity evoked by a common video stimulus. The neural reliability, as quantified by ISC, has been linked to engagement and attentional modulation in earlier studies that used high-grade equipment in laboratory settings. Here we reproduce many of the results from these studies using portable low-cost equipment, focusing on the robustness of using ISC for subjects experiencing naturalistic stimuli. The present data shows that stimulus-ev…
Brain decoding, understood as the process of mapping brain activities to the stimuli that generated them, has been an active research area in the last years. In the case of language stimuli, recent studies have shown that it is possible to decode fMRI scans into an embedding of the word a subject is reading. However, such word embeddings are designed for natural language processing tasks rather than for brain decoding. Therefore, they limit our ability to recover the precise stimulus. In this work, we propose to directly classify an fMRI scan, mapping it to the corresponding word within a fixe…
We present a computational framework for analysis and visualization of non-linear functional connectivity in the human brain from resting state functional MRI (fMRI) data for purposes of recovering the underlying network community structure and exploring causality between network components. Our proposed methodology of non-linear mutual connectivity analysis (MCA) involves two computational steps. First, the pair-wise cross-prediction performance between resting state fMRI pixel time series within the brain is evaluated. The underlying network structure is subsequently recovered from the affin…
How does the human brain represent simple compositions of constituents: actors, verbs, objects, directions, and locations? Subjects viewed videos during neuroimaging (fMRI) sessions from which sentential descriptions of those videos were identified by decoding the brain representations based only on their fMRI activation patterns. Constituents (e.g., "fold" and "shirt") were independently decoded from a single presentation. Independent constituent classification was then compared to joint classification of aggregate concepts (e.g., "fold-shirt"); results were similar as measured by accuracy an…
University of Rajshahi（拉杰沙希大学）；Hajee Mohammad Danesh Science and Technology University（哈吉·穆罕默德·达内什科技大学）；Deakin University（迪肯大学）；COMSATS University Islamabad（伊斯兰堡COMSATS大学）；Charles Sturt University（查尔斯特大学）
In the last two decades, functional magnetic resonance imaging (fMRI) has emerged as one of the most effective technologies in clinical research of the human brain. fMRI allows researchers to study healthy and pathological brains while they perform various neuropsychological functions. Beyond task-related activations, the human brain has some intrinsic activity at a task-negative (resting) state that surprisingly consumes a lot of energy to support communication among neurons. Recent neuroimaging research has also seen an increase in modeling and analyzing brain activity in terms of a graph or…
Florida State University（佛罗里达州立大学）；University of Rochester Medical Center（罗切斯特大学医学中心）
This paper studies change-points in human brain functional connectivity (FC) and seeks patterns that are common across multiple subjects under identical external stimulus. FC relates to the similarity of fMRI responses across different brain regions when the brain is simply resting or performing a task. While the dynamic nature of FC is well accepted, this paper develops a formal statistical test for finding {\it change-points} in times series associated with FC. It represents short-term connectivity by a symmetric positive-definite matrix, and uses a Riemannian metric on this space to develop…
