从非侵入式脑记录中解码语音感知
Decoding speech perception from non-invasive brain recordings
AI总结：
本研究提出一种基于对比学习的模型，可从非侵入式脑磁/脑电图记录中解码感知语音，在多数据集上实现了较高识别准确率，为无创脑语言解码开辟了新路径。
摘要：
从大脑活动中解码语音是医疗健康和神经科学领域期待已久的目标。近年来，侵入式设备在这一领域取得了重要里程碑：基于颅内记录训练的深度学习算法现已开始解码基础语言特征（例如字母、单词、语谱图）。然而，将该方法拓展至自然语音和非侵入式脑记录仍是一项重大挑战。\n本文提出一种采用对比学习训练的模型，可从大规模健康受试者队列的非侵入式记录中解码感知语音的自监督表征。为评估该方法，我们整理并整合了四个公开数据集，涵盖175名接受脑磁图或脑电图（M/EEG）记录的志愿者，记录在他们听短篇故事和孤立句子时进行。\n结果显示，从3秒的MEG信号中，我们的模型能够识别对应的语音片段，在超过1000种不同候选选项中，受试者平均准确率最高可达41%，表现最优的受试者准确率超过80%——该性能可解码训练集中未出现的单词和短语。将我们的模型与多种基线进行比较，凸显了三个关键因素的重要性：（i）对比目标函数，（ii）预训练语音表征，（iii）在多名受试者间同时训练的通用卷积架构。最后，对解码器预测结果的分析表明，其预测主要依赖词汇和语境语义表征。\n总体而言，这种从非侵入式记录中有效解码感知语音的方法，为从大脑活动中解码语言勾勒出了一条前景光明的路径，且无需让患者承担脑部手术的风险。
英文摘要：
Decoding speech from brain activity is a long-awaited goal in both healthcare and neuroscience. Invasive devices have recently led to major milestones in that regard: deep learning algorithms trained on intracranial recordings now start to decode elementary linguistic features (e.g. letters, words, spectrograms). However, extending this approach to natural speech and non-invasive brain recordings remains a major challenge. Here, we introduce a model trained with contrastive-learning to decode self-supervised representations of perceived speech from the non-invasive recordings of a large cohort of healthy individuals. To evaluate this approach, we curate and integrate four public datasets, encompassing 175 volunteers recorded with magneto- or electro-encephalography (M/EEG), while they listened to short stories and isolated sentences. The results show that our model can identify, from 3 seconds of MEG signals, the corresponding speech segment with up to 41% accuracy out of more than 1,000 distinct possibilities on average across participants, and more than 80% in the very best participants - a performance that allows the decoding of words and phrases absent from the training set. The comparison of our model to a variety of baselines highlights the importance of (i) a contrastive objective, (ii) pretrained representations of speech and (iii) a common convolutional architecture simultaneously trained across multiple participants. Finally, the analysis of the decoder's predictions suggests that they primarily depend on lexical and contextual semantic representations. Overall, this effective decoding of perceived speech from non-invasive recordings delineates a promising path to decode language from brain activity, without putting patients at risk for brain surgery.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
