发表机构
Sigma Nova Paris; Goethe University Frankfurt（巴黎西格玛诺瓦公司; 法兰克福歌德大学）
机构由 AI 辅助整理，请以论文原文为准。AI 中文总结
研究针对EEG解码，开发了含多尺度时间卷积输入层和Transformer编码器块的对比预训练模型CoCoT，在广泛基准解码任务中表现出色，优于单任务解码模型，证明对比学习构建EEG基础模型可行，还给出架构设计考量。
AI 中文摘要
自监督预训练基础模型在无创脑电图（EEG）解码应用中已初现成效。近期许多大规模模型采用对原始EEG进行分词，再进行掩码重建预训练的方法。但该方法对EEG这类高噪声幅度、信息局限于窄频带等有限维度的数据并非最优。基于此，我们开发了具有多尺度时间卷积输入层和Transformer编码器块的新型对比预训练EEG模型（CoCoT）。在具有异构电极配置的广泛基准解码任务中，CoCoT与当前最优的重建预训练EEG模型相当或更优。从头开始训练的CoCoT优于先前的单任务解码模型，甚至可与预训练模型媲美，展示了该架构的灵活性和数据效率。通过系统消融实验，我们证明了对比学习构建EEG基础模型的可行性，并提出关键架构设计考量，促使对替代大规模预训练策略进行进一步研究。
英文摘要
Self-supervised pretrained foundation models (FM) have shown early promise for non-invasive electroencephalogram (EEG) decoding applications. Many recent large-scale models converged on the approach of tokenizing raw EEG followed by masked reconstruction pretraining. However, this recipe has been shown to be suboptimal for data, like EEG, with high noise amplitude and information confined to limited dimensions such as narrow frequency bands. Building on this insight, we develop a novel contrastive-pretrained EEG model with multiscale temporal convolution input layers and Transformer encoder blocks (CoCoT). CoCoT matches or beats state-of-the-art reconstruction-pretrained EEG models on extensive benchmark decoding tasks with heterogeneous electrode configurations. Furthermore, CoCoT trained from scratch outperforms previous single-task decoding models and even rivals pretrained models, showcasing the architecture's flexibility and data efficiency. Through systematic ablations, including model architecture and pretraining objective, we demonstrate the viability of contrastive learning for building EEG FMs while suggesting key architectural design considerations, prompting further investigations in alternative large-scale pretraining strategies.
