一便士换你的想法：从廉价脑信号中解码语音
A Penny for Your Thoughts: Decoding Speech from Inexpensive Brain Signals
AI总结：
本文研究利用EEG信号解码语音，通过在Meta解码器上引入受试者特定注意力、个性化空间注意力和双路径RNN等修改，验证了个性化架构对脑到语音解码的改进潜力。
摘要：
我们探索神经网络是否能够通过将脑电图（EEG）记录映射到音频表示，从而将大脑活动解码为语音。利用受试者在聆听自然语音时记录的EEG数据，我们训练了一个模型，采用对比式CLIP损失，将EEG派生的嵌入与来自预训练Transformer语音模型的嵌入进行对齐。在Meta最先进的EEG解码器基础上，我们引入了三项架构修改：（i）受试者特定注意力层（词错误率WER改善+0.15%），（ii）个性化空间注意力（+0.45%），以及（iii）带注意力的双路径RNN（-1.87%）。三项修改中有两项提升了性能，突显了个性化架构在大脑到语音解码及脑机接口应用中的前景。
英文摘要：
We explore whether neural networks can decode brain activity into speech by mapping EEG recordings to audio representations. Using EEG data recorded as subjects listened to natural speech, we train a model with a contrastive CLIP loss to align EEG-derived embeddings with embeddings from a pre-trained transformer-based speech model. Building on the state-of-the-art EEG decoder from Meta, we introduce three architectural modifications: (i) subject-specific attention layers (+0.15% WER improvement), (ii) personalized spatial attention (+0.45%), and (iii) a dual-path RNN with attention (-1.87%). Two of the three modifications improved performance, highlighting the promise of personalized architectures for brain-to-speech decoding and applications in brain-computer interfaces.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
