Functional magnetic resonance imaging (fMRI) is a neuroimaging modality that captures the blood oxygen level in a subject's brain while the subject either rests or performs a variety of functional tasks under different conditions. Given fMRI data, the problem of inferring the task, known as task state decoding, is challenging due to the high dimensionality (hundreds of million sampling points per datum) and complex spatio-temporal blood flow patterns inherent in the data. In this work, we propose to tackle the fMRI task state decoding problem by casting it as a 4D spatio-temporal classification problem. We present a novel architecture called Brain Attend and Decode (BAnD), that uses residual convolutional neural networks for spatial feature extraction and self-attention mechanisms for temporal modeling. We achieve significant performance gain compared to previous works on a 7-task benchmark from the large-scale Human Connectome Project-Young Adult (HCP-YA) dataset. We also investigate the transferability of BAnD's extracted features on unseen HCP tasks, either by freezing the spatial feature extraction layers and retraining the temporal model, or finetuning the entire model. The pre-trained features from BAnD are useful on similar tasks while finetuning them yields competitive results on unseen tasks/conditions.
Decoding Visual Experience and Mapping Semantics through Whole-Brain Analysis Using fMRI Foundation Models
Department of Psychiatry and Behavioral Sciences, Stanford University(精神病学与行为科学系,斯坦福大学);Department of Computer Science, Stanford University(计算机科学系,斯坦福大学);Department of Neurobiology, University of Chicago(神经生物学系,芝加哥大学);Google DeepMind(谷歌DeepMind)
MIRAGE: Robust multi-modal architectures translate fMRI-to-image models from vision to mental imagery
University of Minnesota(明尼苏达大学);Medical AI Research Center (MedARC)(医学人工智能研究中心 (MedARC));University of Sydney(悉尼大学);Stanford University(斯坦福大学);Alljoined;Sophont;Princeton Neuroscience Institute(普林斯顿神经科学研究所)
Select, Attend, and Transfer: Light, Learnable Skip Connections
Simon Fraser University(西蒙弗雷泽大学);Siemens Healthineers(西门子医疗)
深度网络中的跳过连接通过促进更深层次网络架构的训练和降低梯度消失的风险,提高了分割和分类性能。它们为类似编码器-解码器的网络提供了更丰富的特征表示,但代价是更高的内存使用率,计算量,并可能导致传输非歧视性特征图。在本文中,我们专注于改进分段网络中使用的跳过连接(例如,U-Net、V-Net和The One Hundred Layers Tiramisu(DensNet)架构)。我们提出了轻,可学习的跳过连接,学习首先选择最具鉴别力的通道,然后参加所选特征图的最具鉴别力的区域。所提出的跳跃连接的输出是一个独特的特征图,这不仅在很大程度上减少了内存使用和网络参数,但也提高了分割精度。我们在三个不同的2D和体积数据集上评估了所提出的方法,并证明了所提出的轻型,可学习的跳过连接在分割精度,内存使用和网络参数数量方面优于传统的重型跳过连接。
Whole-brain Transferable Representations from Large-Scale fMRI Data Improve Task-Evoked Brain Activity Decoding
Gamania Digital Entertainment Co., Ltd.(Gamania数字娱乐有限公司);Institute of Information Science, Academia Sinica(Academia Sinica信息科学研究所);Sony Computer Science Laboratories, Inc.(索尼计算机科学实验室有限公司)
Spec2VolCAMU-Net: A Spectrogram-to-Volume Model for EEG-to-fMRI Reconstruction based on Multi-directional Time-Frequency Convolutional Attention Encoder and Vision-Mamba U-Net
School of Artificial Intelligence, Chongqing University of Technology(人工智能学院,重庆理工大学)
Animate Your Thoughts: Decoupled Reconstruction of Dynamic Natural Vision from Slow Brain Activity
Institute of Automation, Chinese Academy of Sciences(中国科学院自动化研究所);University of Chinese Academy of Sciences(中国科学院大学);Zhengzhou University(郑州大学);Beijing University of Posts and Telecommunications(北京邮电大学)
See Through Their Minds: Learning Transferable Neural Representation from Cross-Subject fMRI
National Key Laboratory of Human-Machine Hybrid Augmented Intelligence(人机混合增强智能全国重点实验室);National Engineering Research Center of Visual Information and Applications(视觉信息与应用国家工程研究中心);Institute of Artificial Intelligence and Robotics, Xi’an Jiaotong University(西安交通大学人工智能与机器人研究所);Institute of Automation, Chinese Academy of Sciences(中国科学院自动化研究所);University of Chinese Academy of Sciences(中国科学院大学)
Real-time Reconstruction of Human Visual Perception from fMRI
Princeton University(普林斯顿大学);Dartmouth College(达特茅斯学院);Icahn School of Medicine at Mount Sinai(西奈山伊坎医学院);Harvard University(哈佛大学);University of Texas at Dallas(德克萨斯大学达拉斯分校);Medical AI Research Center (MedARC)(医学人工智能研究中心)
Reverse the auditory processing pathway: Coarse-to-fine audio reconstruction from fMRI
Key Laboratory of Brain Cognition and Brain-inspired Intelligence Technology(脑认知与类脑智能技术重点实验室);State Key Laboratory of Multimodal Artificial Intelligence Systems(多模态人工智能系统国家重点实验室);School of Future Technology, University of Chinese Academy of Sciences(中国科学院大学未来技术学院)
Cornell University(康奈尔大学);Weill Cornell Medicine(威尔康奈尔医学院);Brain and Mind Research Institute(脑与思维研究所);Nancy E. and Peter C. Meinig School of Biomedical Engineering(南希·E·和彼得·C·迈尼希生物医学工程学院)
StimFlow: Synthesizing Brain Activity from Images via Stimulus-conditioned Flow Matching
Chonnam National University(全南大学);AISEED, Inc.(AISEED公司);Ho Chi Minh University of Science(胡志明市科学大学);MIREA – Russian Technological University(俄罗斯国立研究型技术大学(莫斯科国立精密机械与计算机科学研究所))
TReND: Transformer derived features and Regularized NMF for neonatal functional network Delineation
University of Pennsylvania(宾夕法尼亚大学);Children’s Hospital of Philadelphia(费城儿童医院);Peking University(北京大学);Beijing Children’s Hospital(北京儿童医院);Beijing Normal University(北京师范大学)