PolyU；South China University of Technology（南方科技大学）；Tsinghua University（清华大学）；YMSC, Tsinghua University（清华大学交叉信息学院）；Dept. of Biomedical Engineering, PolyU（PolyU 生物医学工程系）；Sch. of Future Technology, South China University of Technology（南方科技大学未来技术学院）；Dept. of Health Technology and Informatics, PolyU（PolyU 健康技术与信息学系）；Dept. of Biomedical Engineering and the Dept. of Data Science and Artificial Intelligence, PolyU（PolyU 生物医学工程系和数据科学与人工智能系）
Guoqi Yu, Xiaowei Hu, Angelica I. Aviles-Rivero, Anqi Qiu, Shujun Wang
备注：
This paper has been accepted by IEEE Transactions on Medical Imaging
Functional magnetic resonance imaging (fMRI) enables non-invasive brain disorder classification by capturing blood-oxygen-level-dependent (BOLD) signals. However, most existing methods rely on functional connectivity (FC) via Pearson correlation, which reduces 4D BOLD signals to static 2D matrices, discarding temporal dynamics and capturing only linear inter-regional relationships. In this work, we benchmark state-of-the-art temporal models (e.g., time-series models such as PatchTST, TimesNet, and TimeMixer) on raw BOLD signals across five public datasets. Results show these models consistently outperform traditional FC-based approaches, highlighting the value of directly modeling temporal information such as cycle-like oscillatory fluctuations and drift-like slow baseline trends. Building on this insight, we propose DeCI, a simple yet effective framework that integrates two key principles: (i) Cycle and Drift Decomposition to disentangle cycle and drift within each ROI (Region of Interest); and (ii) Channel-Independence to model each ROI separately, improving robustness and reducing overfitting. Extensive experiments demonstrate that DeCI achieves superior classification accuracy and generalization compared to both FC-based and temporal baselines. Our findings advocate for a shift toward end-to-end temporal modeling in fMRI analysis to better capture complex brain dynamics. The code is available at https://github.com/Levi-Ackman/DeCI.
fMRI-PTE: A Large-scale fMRI Pretrained Transformer Encoder for Multi-Subject Brain Activity Decoding
School of Data Science, Fudan University（复旦大学数据科学学院）；Institute of Science and Technology for Brain-inspired Intelligence, Fudan University（复旦大学类脑智能科学与技术研究院）
MindAligner: Explicit Brain Functional Alignment for Cross-Subject Visual Decoding from Limited fMRI Data
Shanghai Artificial Intelligence Laboratory（上海人工智能实验室）；Nanjing University of Science and Technology（南京理工大学）；The Chinese University of Hong Kong（香港中文大学）；Tsinghua University（清华大学）
Seoul National University（首尔大学）；SungKyunKwan University（成均馆大学）；Brookhaven National Lab（布鲁克海文国家实验室）
Modeling spatiotemporal brain dynamics from high-dimensional data, such as functional Magnetic Resonance Imaging (fMRI), is a formidable task in neuroscience. Existing approaches for fMRI analysis utilize hand-crafted features, but the process of feature extraction risks losing essential information in fMRI scans. To address this challenge, we present SwiFT (Swin 4D fMRI Transformer), a Swin Transformer architecture that can learn brain dynamics directly from fMRI volumes in a memory and computation-efficient manner. SwiFT achieves this by implementing a 4D window multi-head self-attention mec…
A Survey on fMRI-based Brain Decoding for Reconstructing Multimodal Stimuli
Beijing Institute of Basic Medical Sciences（北京基础医学研究所）；School of Computer Science and Information Engineering, Hefei University of Technology (HFUT)（合肥工业大学计算机与信息工程学院）；Institute of Artificial Intelligence, Hefei Comprehensive National Science Center（合肥综合性国家科学中心人工智能研究院）；Zhejiang University（浙江大学）；Australian Artificial Intelligence Institute, Faculty of Engineering and Information Technology, University of Technology Sydney（悉尼科技大学工程与信息技术学院澳大利亚人工智能研究所）；School of Information Science and Technology, University of Science and Technology of China（中国科学技术大学信息科学技术学院）
BrainWorld: A Structural-Prior-Conditioned Generative Model for Whole-Brain 4D fMRI Dynamics
Department of Biomedical Engineering, Southern University of Science and Technology（南方科技大学生物医学工程系）；School of Biomedical Engineering, Shenzhen University（深圳大学生物医学工程学院）
Chinese PLA General Hospital（中国人民解放军总医院）；Shenzhen Institutes of Advanced Technology, Chinese Academy of Sciences（中国科学院深圳先进技术研究院）；Peng Cheng Laboratory（鹏城实验室）；Beijing Jiaotong University（北京交通大学）；Institute of Computing Technology, Chinese Academy of Sciences（中国科学院计算技术研究所）
In neuroimaging analysis, fMRI can well assess the function changes for brain diseases with no obvious structural lesions. To date, most deep-learning-based fMRI studies have employed functional connectivity (FC) as the basic feature for disease classification. However, FC is calculated on time series of predefined regions of interest and neglects detailed information contained in each voxel. Another drawback of using FC is the limited sample size for the training of deep models. The low representation ability of FC leads to poor performance in clinical practice, especially when dealing with m…
Decoding Visual Experience and Mapping Semantics through Whole-Brain Analysis Using fMRI Foundation Models
Department of Psychiatry and Behavioral Sciences, Stanford University（精神病学与行为科学系，斯坦福大学）；Department of Computer Science, Stanford University（计算机科学系，斯坦福大学）；Department of Neurobiology, University of Chicago（神经生物学系，芝加哥大学）；Google DeepMind（谷歌DeepMind）
Columbia University（哥伦比亚大学）；New York State Psychiatric Institute（纽约州精神病学研究所）
The goal of emotional brain state classification on functional MRI (fMRI) data is to recognize brain activity patterns related to specific emotion tasks performed by subjects during an experiment. Distinguishing emotional brain states from other brain states using fMRI data has proven to be challenging due to two factors: a difficulty to generate fast yet accurate predictions in short time frames, and a difficulty to extract emotion features which generalize to unseen subjects. To address these challenges, we conducted an experiment in which 22 subjects viewed pictures designed to stimulate ei…
BrainCast: A Spatio-Temporal Forecasting Model for Whole-Brain fMRI Time Series Prediction
Institute of Advanced Technology, University of Science and Technology of China（科学技术大学先进技术研究院）；MoE Key Laboratory of Brain-Inspired Intelligence Perception and Cognition, University of Science and Technology of China（科学技术大学脑启发智能感知与认知教育部重点实验室）；School of Mathematics and Computer Sciences, Nanchang University（南昌大学数学与计算机科学学院）；School of Data Science, University of Virginia（弗吉尼亚大学数据科学学院）；Department of Biomedical Engineering, Tulane University（路易斯安那大学医学工程系）
Whole-brain Transferable Representations from Large-Scale fMRI Data Improve Task-Evoked Brain Activity Decoding
Gamania Digital Entertainment Co., Ltd.（Gamania数字娱乐有限公司）；Institute of Information Science, Academia Sinica（Academia Sinica信息科学研究所）；Sony Computer Science Laboratories, Inc.（索尼计算机科学实验室有限公司）
Generative AI Enables Structural Brain Network Construction from fMRI via Symmetric Diffusion Learning
Department of Computing, Hong Kong Polytechnic University（香港理工大学计算学系）；School of Information Engineering, Nanchang University（南昌大学信息工程学院）；Shenzhen Institutes of Advanced Technology, Chinese Academy of Sciences（中国科学院深圳先进技术研究院）
Brain-DiT: A Universal Multi-state fMRI Foundation Model with Metadata-Conditioned Pretraining
Department of Biomedical Engineering, Southern University of Science and Technology, China（南方科技大学生物医学工程系，中国）；School of Biomedical Engineering, Shenzhen University, China（深圳大学生物医学工程学院，中国）
University of Rome, Tor Vergata（罗马第二大学）；Istituto di Bioimmagini e Fisiologia Molecolare, CNR（意大利国家研究委员会生物成像与分子生理学研究所）；Martinos Center for Biomedical Imaging（马丁诺斯生物医学成像中心）；MGH（麻省总医院）；Harvard Medical School（哈佛医学院）
Brain decoding is a field of computational neuroscience that uses measurable brain activity to infer mental states or internal representations of perceptual inputs. Therefore, we propose a novel approach to brain decoding that also relies on semantic and contextual similarity. We employ an fMRI dataset of natural image vision and create a deep learning decoding pipeline inspired by the existence of both bottom-up and top-down processes in human vision. We train a linear brain-to-feature model to map fMRI activity features to visual stimuli features, assuming that the brain projects visual info…
In the past few years, significant advancements were made in reconstruction of observed natural images from fMRI brain recordings using deep-learning tools. Here, for the first time, we show that dense 3D depth maps of observed 2D natural images can also be recovered directly from fMRI brain recordings. We use an off-the-shelf method to estimate the unknown depth maps of natural images. This is applied to both: (i) the small number of images presented to subjects in an fMRI scanner (images for which we have fMRI recordings - referred to as "paired" data), and (ii) a very large number of natura…
4D iterative reconstruction of brain fMRI in the moving fetus
Computational Imaging Research Lab（计算成像研究实验室）；Medical University of Vienna（维也纳医科大学）；Lausanne University Hospital (CHUV)（洛桑大学医院（CHUV））；University of Lausanne (UNIL)（洛桑大学（UNIL））；CIBM Center for Biomedical Imaging（CIBM生物医学成像中心）
Hi-DREAM: Brain-Inspired Hierarchical Diffusion for fMRI-to-Image Reconstruction via ROI Encoder and VisuAl Mapping
Dept of Data Science and AI, Monash University, Melbourne, Australia（数据科学与人工智能系，墨尔本大学，澳大利亚）；School of Psychological Sciences, Monash University, Melbourne, Australia（心理学科学学院，墨尔本大学，澳大利亚）；Brain and Mind Centre, University of Sydney, Sydney, Australia（脑与心智中心，悉尼大学，澳大利亚）
Tokyo Institute of Technology（东京工业大学）；National Institute of Advanced Industrial Science and Technology（产业技术综合研究所）；AIST-Tokyo Tech Real World Big-Data Computation Open Innovation Laboratory（AIST-东京工业大学真实世界大数据计算开放创新实验室）
With the advent of brain imaging techniques and machine learning tools, much effort has been devoted to building computational models to capture the encoding of visual information in the human brain. One of the most challenging brain decoding tasks is the accurate reconstruction of the perceived natural images from brain activities measured by functional magnetic resonance imaging (fMRI). In this work, we survey the most recent deep learning methods for natural image reconstruction from fMRI. We examine these methods in terms of architectural design, benchmark datasets, and evaluation metrics …
Reconstructing natural videos from fMRI brain recordings is very challenging, for two main reasons: (i) As fMRI data acquisition is difficult, we only have a limited amount of supervised samples, which is not enough to cover the huge space of natural videos; and (ii) The temporal resolution of fMRI recordings is much lower than the frame rate of natural videos. In this paper, we propose a self-supervised approach for natural-movie reconstruction. By employing cycle-consistency over Encoding-Decoding natural videos, we can: (i) exploit the full framerate of the training videos, and not be limit…
Korea Advanced Institute of Science and Technology (KAIST)（韩国科学技术院）
Graph neural networks (GNN) rely on graph operations that include neural network training for various graph related tasks. Recently, several attempts have been made to apply the GNNs to functional magnetic resonance image (fMRI) data. Despite recent progresses, a common limitation is its difficulty to explain the classification results in a neuroscientifically explainable way. Here, we develop a framework for analyzing the fMRI data using the Graph Isomorphism Network (GIN), which was recently proposed as a powerful GNN for graph classification. One of the important contributions of this paper…
Stacked Regression using Off-the-shelf, Stimulus-tuned and Fine-tuned Neural Networks for Predicting fMRI Brain Responses to Movies (Algonauts 2025 Report)
Université Paris Cité（巴黎Cité大学）；University of Oxford（牛津大学）；University of Turin（都灵大学）；Universität Leipzig（莱比锡大学）；Max Planck School of Cognition（马克斯·普朗克认知科学学院）
Reconstructing perceived natural images from fMRI signals is one of the most engaging topics of neural decoding research. Prior studies had success in reconstructing either the low-level image features or the semantic/high-level aspects, but rarely both. In this study, we utilized an Instance-Conditioned GAN (IC-GAN) model to reconstruct images from fMRI patterns with both accurate semantic attributes and preserved low-level details. The IC-GAN model takes as input a 119-dim noise vector and a 2048-dim instance feature vector extracted from a target image via a self-supervised learning model (…
