发表机构
University of the Chinese Academy of Sciences; Institute of Software, Chinese Academy of Sciences; Beijing Language and Culture University（中国科学院大学; 中国科学院软件研究所; 北京语言大学）
机构由 AI 辅助整理，请以论文原文为准。AI 中文总结
研究针对从头皮脑电图解码中文语音信息的难题，提出EEGAlign框架，通过联合文本与音频对齐及对比学习，经CTC解码，在ChineseEEG-2数据上取得先进的闭集句子分类性能，证明两对齐轴互补，是该领域首次相关研究。
AI 中文摘要
直接从头皮脑电图（EEG）解码语音信息，为严重言语和运动障碍患者提供了潜在的非侵入性神经通信途径。与侵入性方法相比，EEG更安全且可广泛部署，但对中文句子解码更具挑战性。现有方法仅专注单一监督轴，无法同时满足大词汇量中文解码的句子级可辨别性和细粒度时间分辨率要求。我们引入EEGAlign，通过对比学习将EEG与文本和音频对齐，再经CTC字符序列解码。在ChineseEEG-2数据上，EEGAlign取得了先进的闭集句子分类性能。消融研究表明两个对齐轴高度互补。这是首次在公开语音产生过程中从非侵入性EEG解码大词汇量中文句子并在相对大的闭集候选句子设置下取得强大分类性能的研究。
英文摘要
Decoding speech information directly from scalp electroencephalography (EEG) into text provides a potential non-invasive neural communication pathway for individuals with severe speech and motor impairments. Compared with invasive approaches such as electrocorticography, EEG is safer and more widely deployable, yet substantially more challenging to decode.This challenge is exacerbated for Chinese sentence decoding, which must handle a high-dimensional output space with thousands of characters, severe inter-subject variability, and low signal-to-noise ratios for text alignment.Existing methods commit to a single supervisory axis---either text semantics or audio acoustic features---yet neither can simultaneously satisfy the demands of sentence-level discriminability and fine-grained temporal resolution required for large-vocabulary Chinese decoding. We introduce EEGAlign, a novel parameter-efficient framework that jointly aligns EEG with two axes---text alignment with BGE-M3 text embeddings and audio alignment with wav2vec~2.0 speech features via contrastive learning followed by CTC character-sequence decoding. On ChineseEEG-2 data, EEGAlign yields state-of-the-art closed-set sentence classification performance, reaching up to 82.37% Top-1 accuracy on Reading Aloud EEG and 41.43% on Passive Listening EEG out of 101 candidates. Ablation studies show that the two alignment axes are highly complementary: combining them yields consistently better performance than either alone. To the best of our knowledge, this is the first study on decoding large-vocabulary Chinese sentences from non-invasive EEG during overt speech production, and achieving strong classification performance with relatively large closed-set candidate-sentence setting.
