临床脑-文本接口旨在为无法提供大量训练记录的瘫痪患者设计。预训练通过学习跨受试者的统计先验提升数据效率，但这些先验依赖于上下文。尽管自然语音可能持续数分钟，大多数方法仅使用几秒的上下文进行预训练。因此，我们提出MEG-XL，一种预训练时每个样本使用2.5分钟MEG上下文的模型，比以往工作长5-300倍，相当于191k个token，捕捉到扩展的神经上下文。在从脑数据解码词的任务中微调，MEG-XL在少量数据下（例如1小时 vs 50小时）达到监督性能，并优于脑基础模型。我们发现预训练时使用更长上下文的模型在转移到词解码任务时表现更好。我们的结果表明，长上下文预训练有助于利用其他方法不必要的丢弃的扩展神经上下文。代码、模型权重和说明可在https://github.com/neural-processing-lab/MEG-XL获取。
英文摘要：
Clinical brain-to-text interfaces are designed for paralysed patients who cannot provide extensive training recordings. Pre-training improves data-efficient generalisation by learning statistical priors across subjects, but these priors critically depend on context. While natural speech might unfold gradually over minutes, most methods pre-train with only a few seconds of context. Thus, we propose MEG-XL, a model pre-trained with 2.5 minutes of MEG context per sample, 5-300x longer than prior work, and equivalent to 191k tokens, capturing extended neural context. Fine-tuning on the task of word decoding from brain data, MEG-XL matches supervised performance with a fraction of the data (e.g. 1hr vs 50hrs) and outperforms brain foundation models. We find that models pre-trained with longer contexts learn representations that transfer better to word decoding. Our results indicate that long-context pre-training helps exploit extended neural context that other methods unnecessarily discard. Code, model weights, and instructions are available at https://github.com/neural-processing-lab/MEG-XL .
Neuroimaging techniques have shown to be useful when studying the brain's activity. This paper uses Magnetoencephalography (MEG) data, provided by the Human Connectome Project (HCP), in combination with various deep artificial neural network models to perform brain decoding. More specifically, here we investigate to which extent can we infer the task performed by a subject based on its MEG data. Three models based on compact convolution, combined convolutional and long short-term architecture as well as a model based on multi-view learning that aims at fusing the outputs of the two stream netw…
BrainOmni: A Brain Foundation Model for Unified EEG and MEG Signals
Shanghai Artificial Intelligence Laboratory, China（上海人工智能实验室）；Department of Electronic Engineering, Tsinghua University, China（清华大学电子工程系）；Department of Psychology, University of Cambridge, UK（剑桥大学心理学系）；Speech Hearing and Phonetic Sciences, University College London, UK（伦敦大学学院语音听力与语音科学系）；MRC Cognition and Brain Sciences Unit, University of Cambridge, UK（剑桥大学MRC认知与脑科学单位）
McGovern Institute for Brain Research, Massachusetts Institute of Technology（麻省理工学院麦戈文脑研究所）；Complutense University of Madrid（马德里康普顿斯大学）；Harvard Medical School（哈佛医学院）
Characterizing the subtle changes of functional brain networks associated with the pathological cascade of Alzheimer's disease (AD) is important for early diagnosis and prediction of disease progression prior to clinical symptoms. We developed a new deep learning method, termed multiple graph Gaussian embedding model (MG2G), which can learn highly informative network features by mapping high-dimensional resting-state brain networks into a low-dimensional latent space. These latent distribution-based embeddings enable a quantitative characterization of subtle and heterogeneous brain connectivit…
Department of Neuroscience and Biomedical Engineering, Aalto University School of Science（阿尔托大学理学院神经科学与生物医学工程系）；Aalto NeuroImaging, Aalto University（阿尔托大学阿尔托神经成像中心）；Aalto University School of Science（阿尔托大学理学院）；Aalto University（阿尔托大学）
Convolutional Neural Networks (CNN) outperform traditional classification methods in many domains. Recently these methods have gained attention in neuroscience and particularly in brain-computer interface (BCI) community. Here, we introduce a CNN optimized for classification of brain states from magnetoencephalographic (MEG) measurements. Our CNN design is based on a generative model of the electromagnetic (EEG and MEG) brain signals and is readily interpretable in neurophysiological terms. We show here that the proposed network is able to decode event-related responses as well as modulations …
How meaning is represented in the brain is still one of the big open questions in neuroscience. Does a word (e.g., bird) always have the same representation, or does the task under which the word is processed alter its representation (answering "can you eat it?" versus "can it fly?")? The brain activity of subjects who read the same word while performing different semantic tasks has been shown to differ across tasks. However, it is still not understood how the task itself contributes to this difference. In the current work, we study Magnetoencephalography (MEG) brain recordings of participants…
Directorate of Technology and Innovation（技术与创新局）；Chamber of Deputies（众议院）
Deep predictive coding networks are neuroscience-inspired unsupervised learning models that learn to predict future sensory states. We build upon the PredNet implementation by Lotter, Kreiman, and Cox (2016) to investigate if predictive coding representations are useful to predict brain activity in the visual cortex. We use representational similarity analysis (RSA) to compare PredNet representations to functional magnetic resonance imaging (fMRI) and magnetoencephalography (MEG) data from the Algonauts Project. In contrast to previous findings in the literature (Khaligh-Razavi &Kriegeskorte, …
Massachusetts General Hospital（麻省总医院）；Harvard Medical School（哈佛医学院）；Picower Institute for Learning and Memory（皮考尔学习与记忆研究所）；MIT（麻省理工学院）；The University of Texas at Arlington（德克萨斯大学阿灵顿分校）；The University of Texas at Dallas（德克萨斯大学达拉斯分校）
Brain source imaging is an important method for noninvasively characterizing brain activity using Electroencephalogram (EEG) or Magnetoencephalography (MEG) recordings. Traditional EEG/MEG Source Imaging (ESI) methods usually assume that either source activity at different time points is unrelated, or that similar spatiotemporal patterns exist across an entire study period. The former assumption makes ESI analyses sensitive to noise, while the latter renders ESI analyses unable to account for time-varying patterns of activity. To effectively deal with noise while maintaining flexibility and co…
Towards improving Alzheimer's intervention: a machine learning approach for biomarker detection through combining MEG and MRI pipelines
Universiti Teknologi PETRONAS（马来西亚国油大学）；Hospital Universiti Sains Malaysia（马来西亚理科大学医院）；Universiti Sains Malaysia（马来西亚理科大学）；Ulster University（阿尔斯特大学）；University of Calgary（卡尔加里大学）；University of Bath（巴斯大学）
本文基于 324 名 BioFIND 参与者的 MEG 与 MRI 数据，通过交叉验证机器学习分类识别 MCI，发现融合未校正 MEG 与 z-score 校正 MRI 特征效果最佳。
MEG-GPT: A transformer-based foundation model for magnetoencephalography data
Oxford Centre for Integrative Neuroimaging (OxCIN)（牛津整合神经影像中心）；University of Oxford（牛津大学）；Department of Psychiatry（精神病学系）；Nuffield Department of Clinical Neurosciences（努尔菲尔德临床神经科学系）；Department of Engineering Science（工程科学系）
本文提出 MEG-GPT，一种基于 transformer、时间注意力和下一时间点预测的 MEG 基础模型，并通过无损数据驱动分词器实现真实脑信号生成与跨会话、跨受试者零样本解码提升。
Inserm（法国国家健康与医学研究院）；Paris Diderot University（巴黎第七大学）；Lariboisière Hospital（拉里博瓦西耶医院）；Assistance Publique Hôpitaux de Paris（巴黎公共援助医院）；Université Paris-Saclay（巴黎萨克雷大学）；Inria（法国国家信息与自动化研究所）；CEA（法国原子能和替代能源委员会）
Magnetoencephalography and electroencephalography (M/EEG) can reveal neuronal dynamics non-invasively in real-time and are therefore appreciated methods in medicine and neuroscience. Recent advances in modeling brain-behavior relationships have highlighted the effectiveness of Riemannian geometry for summarizing the spatially correlated time-series from M/EEG in terms of their covariance. However, after artefact-suppression, M/EEG data is often rank deficient which limits the application of Riemannian concepts. In this article, we focus on the task of regression with rank-reduced covariance ma…
MEG-to-MEG Transfer Learning and Cross-Task Speech/Silence Detection with Limited Data
HiTZ Center, University of the Basque Country -- UPV/EHU, Spain（比斯开国家大学HiTZ中心）；Basque Center on Cognition, Brain and Language -- BCBL, Spain（巴斯克认知、大脑与语言中心）；Ikerbasque, Basque Foundation for Science, Spain（巴斯克科学基金会Ikerbasque）；Laboratory of Functional Anatomy, Faculty of Human Motor Sciences, Université libre de Bruxelles (ULB), Belgium（布鲁塞尔自由大学人体运动科学系功能解剖实验室）；Laboratoire de Neuroanatomie et Neuroimagerie translationnelles (LN2T), ULB Neuroscience Institute, Université libre de Bruxelles (ULB), Belgium（布鲁塞尔自由大学神经解剖与神经影像翻译实验室）；WEL Research Institute, Belgium（比利时WEL研究机构）
LTCI, Télécom ParisTech, Université Paris-Saclay（LTCI，巴黎高等电信学院，巴黎萨克雷大学）；INRIA, Université Paris Saclay（法国国家信息与自动化研究所，巴黎萨克雷大学）
Frequency-specific patterns of neural activity are traditionally interpreted as sustained rhythmic oscillations, and related to cognitive mechanisms such as attention, high level visual processing or motor control. While alpha waves (8-12 Hz) are known to closely resemble short sinusoids, and thus are revealed by Fourier analysis or wavelet transforms, there is an evolving debate that electromagnetic neural signals are composed of more complex waveforms that cannot be analyzed by linear filters and traditional signal representations. In this paper, we propose to learn dedicated representations…
Segmentation is one of the most important tasks in MRI medical image analysis and is often the first and the most critical step in many clinical applications. In brain MRI analysis, head segmentation is commonly used for measuring and visualizing the brain's anatomical structures and is also a necessary step for other applications such as current-source reconstruction in electroencephalography and magnetoencephalography (EEG/MEG). Here we propose a deep learning framework that can segment brain, skull, and extra-cranial tissue using only T1-weighted MRI as input. In addition, we describe a rob…
Time CNN and Graph Convolution Network for Epileptic Spike Detection in MEG Data
Lyon Neuroscience Research center（里昂神经科学研究中心）；INSERM（法国国家健康与医学研究院）；Functional Neurology and Epileptogy, HCL（里昂公立医院集团功能神经与癫痫科）；Univ Lyon（里昂大学）；CNRS（法国国家科学研究中心）；INSA Lyon（里昂国立应用科学学院）；UCBL（里昂第一大学）；CREATIS（里昂成像与生物技术研究中心）
Diffusion-based translation between unpaired spontaneous premature neonatal EEG and fetal MEG
Laboratoire de Mathématiques de Lens (LML), UR 2462, Université d'Artois（Lens数学实验室（LML），UR 2462，阿托瓦大学）；Department of Internal Medicine IV, University Hospital of Tübingen（第四内科部门，图宾根大学医院）；IDM/fMEG Center of the Helmholtz Center Munich, University of Tübingen, German Center for Diabetes Research (DZD)（慕尼黑海德堡中心IDM/fMEG中心，图宾根大学，糖尿病研究德国中心（DZD））；Institute of Pharmaceutical Sciences, Department of Pharmacy and Biochemistry, Interfaculty Centre for Pharmacogenomics and Pharma Research, Eberhard Karls University Tübingen（药学研究所，药学与生物化学部门，药基因组学与药学研究跨学科中心，图宾根埃伯哈德·卡尔斯大学）
Decoding Phone Pairs from MEG Signals Across Speech Modalities
HiTZ Center, University of the Basque Country -- UPV/EHU（巴斯克国家大学希茨中心）；Basque Centre on Cognition, Brain and Language -- BCBL（巴斯克认知、大脑与语言中心）；Laboratory of Functional Anatomy, Faculty of Human Motor Sciences, Université libre de Bruxelles (ULB)（人类运动科学系功能解剖实验室）；Laboratoire de Neuroanatomie et Neuroimagerie translationnelles (LN2T), ULB Neuroscience Institute, Université libre de Bruxelles (ULB)（神经解剖与神经影像翻译实验室（LN2T）、ULB神经科学研究所）；WEL Research Institute（WEL研究机构）
Decoding individual words from non-invasive brain recordings across 723 participants
Meta AI；CNRS（法国国家科学研究中心）；INSERM（法国国家健康与医学研究院）；CEA（法国原子能和替代能源委员会）；Neurospin center（神经自旋中心）；École Normale Supérieure（巴黎高等师范学院）；PSL University（巴黎文理研究大学）；Laboratoire des Systèmes Perceptifs（感知系统实验室）
Inria Saclay（法国国家信息与自动化研究所萨克雷中心）；Université de Montpellier（蒙彼利埃大学）；CEA（法国原子能和替代能源委员会）
Detecting where and when brain regions activate in a cognitive task or in a given clinical condition is the promise of non-invasive techniques like magnetoencephalography (MEG) or electroencephalography (EEG). This problem, referred to as source localization, or source imaging, poses however a high-dimensional statistical inference challenge. While sparsity promoting regularizations have been proposed to address the regression problem, it remains unclear how to ensure statistical control of false detections. Moreover, M/EEG source imaging requires to work with spatio-temporal data and autocorr…
Massachusetts General Hospital（麻省总医院）；Harvard Medical School（哈佛医学院）；Harvard-MIT Division of Health Sciences and Technology（哈佛-麻省理工卫生科学与技术部）；Institute for Medical Engineering, Research Campus STIMULATE, Otto-von-Guericke University（奥托·冯·格里克大学医学工程研究所（STIMULATE研究校区））；Institute for Biomedical Engineering and Informatics, Technische Universität Ilmenau（伊尔默瑙工业大学生物医学工程与信息学研究所）
Magnetoencephalography (MEG) and Electroencephalography (EEG) source estimates have thus far mostly been derived sample by sample, i.e., independent of each other in time. However, neuronal assemblies are heavily interconnected, constraining the temporal evolution of neural activity in space as detected by MEG and EEG. The observed neural currents are thus highly context dependent. Here, a new method is presented which integrates predictive deep learning networks with the Minimum-Norm Estimates (MNE) approach. Specifically, we employ Long Short-Term Memory (LSTM) networks, a type of recurrent …
South China University of Technology（华南理工大学）；Stanford University（斯坦福大学）；Chongqing University of Posts and Telecommunications（重庆邮电大学）；Central China Normal University（华中师范大学）；Guangdong University of Finance（广东金融学院）
Electromagnetic source imaging (ESI) requires solving a highly ill-posed inverse problem. To seek a unique solution, traditional ESI methods impose various forms of priors that may not accurately reflect the actual source properties, which may hinder their broad applications. To overcome this limitation, in this paper a novel data-synthesized spatio-temporally convolutional encoder-decoder network method termed DST-CedNet is proposed for ESI. DST-CedNet recasts ESI as a machine learning problem, where discriminative learning and latent-space representations are integrated in a convolutional en…
