面向主动脑机接口的大型脑语言模型预训练：无声语音
Pretraining Large Brain Language Model for Active BCI: Silent Speech
AI总结：
本文提出大型脑语言模型LBLM及未来时频预测FSTP预训练范式，利用120小时EEG无声语音数据实现主动BCI解码，在跨会话词级和语义级分类上显著优于基线方法。
摘要：
本文探索了主动脑机接口（BCI）系统中的无声语音解码，这类系统相比传统BCI应用能够提供更自然、更灵活的通信方式。我们收集了一个新的无声语音数据集，包含来自12名受试者的超过120小时脑电图（EEG）记录，涵盖24个常用英语单词，用于语言模型预训练和解码。鉴于近期利用自监督范式预训练大型模型以提升EEG分类性能的成功经验，我们提出了大型脑语言模型（Large Brain Language Model，LBLM），该模型经过预训练以解码主动BCI中的无声语音。为了预训练LBLM，我们提出了未来时频预测（Future Spectro-Temporal Prediction，FSTP）预训练范式，以从未标注的EEG数据中学习有效表示。与现有主要遵循掩码重建范式的EEG预训练方法不同，我们提出的FSTP方法在时间域和频率域中采用自回归建模，以同时捕获EEG信号中的时间依赖性和频谱依赖性。预训练之后，我们在下游任务上对LBLM进行微调，包括词级和语义级分类。大量实验表明，LBLM相比全监督和预训练基线模型取得了显著的性能提升。例如，在困难的跨会话设置下，我们的模型在语义级分类上达到47.0%的准确率，在词级分类上达到39.6%的准确率，分别比基线方法高出5.4%和7.3%。我们的研究推动了主动BCI系统中无声语音解码的发展，为EEG语言模型预训练提供了一种创新解决方案，并为基础研究提供了一个新的数据集。
英文摘要：
This paper explores silent speech decoding in active brain-computer interface (BCI) systems, which offer more natural and flexible communication than traditional BCI applications. We collected a new silent speech dataset of over 120 hours of electroencephalogram (EEG) recordings from 12 subjects, capturing 24 commonly used English words for language model pretraining and decoding. Following the recent success of pretraining large models with self-supervised paradigms to enhance EEG classification performance, we propose Large Brain Language Model (LBLM) pretrained to decode silent speech for active BCI. To pretrain LBLM, we propose Future Spectro-Temporal Prediction (FSTP) pretraining paradigm to learn effective representations from unlabeled EEG data. Unlike existing EEG pretraining methods that mainly follow a masked-reconstruction paradigm, our proposed FSTP method employs autoregressive modeling in temporal and frequency domains to capture both temporal and spectral dependencies from EEG signals. After pretraining, we finetune our LBLM on downstream tasks, including word-level and semantic-level classification. Extensive experiments demonstrate significant performance gains of the LBLM over fully-supervised and pretrained baseline models. For instance, in the difficult cross-session setting, our model achieves 47.0\% accuracy on semantic-level classification and 39.6\% in word-level classification, outperforming baseline methods by 5.4\% and 7.3\%, respectively. Our research advances silent speech decoding in active BCI systems, offering an innovative solution for EEG language model pretraining and a new dataset for fundamental research.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
