基于视觉变换器和对比学习的皮层电图信号外显语音解码
Neural Decoding of Overt Speech from ECoG Using Vision Transformers and Contrastive Representation Learning
AI总结：
本文提出基于视觉变换器和对比学习的ECoG信号语音解码方法，通过编码器-解码器架构实现流式语音重建，首次在完全植入式皮层记录系统中展示语音解码的可行性。
摘要：
语音脑机接口（BCIs）为无法交流的严重瘫痪患者提供了有前景的解决方案。近期的一些研究已证明，通过预测一系列音节或单词，并利用下游语言模型生成有意义的句子，可以有效地从皮层电图（ECoG）或皮层内记录中重建可理解的语音。当前的挑战是通过直接回归皮层信号到语音信号来实现流式语音重建。虽然最近使用皮层内数据实现了这一点，但还需要进一步的工作来获得与表面ECoG记录相媲美的结果。特别是，在这种情况下优化神经解码器变得至关重要。本文提出了一种基于编码器-解码器深度神经架构的离线语音解码流程，整合了视觉变换器和对比学习以增强从ECoG信号直接回归语音的能力。该方法在两个数据集上进行了评估，一个数据集是通过临床亚急性电极在癫痫患者身上获得的，另一个数据集是通过完全植入式WIMAGINE皮层系统在一项运动BCI试验参与者身上获得的。据我们所知，这首次尝试了从完全植入式和无线皮层记录系统中解码语音，为长期使用提供了前景。
英文摘要：
Speech Brain Computer Interfaces (BCIs) offer promising solutions to people with severe paralysis unable to communicate. A number of recent studies have demonstrated convincing reconstruction of intelligible speech from surface electrocorticographic (ECoG) or intracortical recordings by predicting a series of phonemes or words and using downstream language models to obtain meaningful sentences. A current challenge is to reconstruct speech in a streaming mode by directly regressing cortical signals into acoustic speech. While this has been achieved recently using intracortical data, further work is needed to obtain comparable results with surface ECoG recordings. In particular, optimizing neural decoders becomes critical in this case. Here we present an offline speech decoding pipeline based on an encoder-decoder deep neural architecture, integrating Vision Transformers and contrastive learning to enhance the direct regression of speech from ECoG signals. The approach is evaluated on two datasets, one obtained with clinical subdural electrodes in an epileptic patient, and another obtained with the fully implantable WIMAGINE epidural system in a participant of a motor BCI trial. To our knowledge this presents a first attempt to decode speech from a fully implantable and wireless epidural recording system offering perspectives for long-term use.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
