面向语音产生过程中超声舌图像从EEG预测的研究
Towards Ultrasound Tongue Image prediction from EEG during speech production
AI总结：
本研究探索从语音产生时的EEG信号预测超声舌图像，采用全连接深度神经网络，发现EEG与舌图像存在微弱但可区分的关系，为语音BCI提供可行性依据。
摘要：
先前已有初步研究提出基于脑信号（如非侵入性脑电图EEG和侵入性皮层脑电图sEEG/ECoG）的语音脑机接口（BCI），但缺乏联合方法同时研究非侵入性脑信号、发音器官运动和语音信号，并分析大脑中的认知过程、发音运动的运动学以及由此产生的语音信号。本文中，我们描述了我们多模态（脑电图、超声舌成像和语音）的分析与合成实验，作为一项可行性研究。我们将语音产生过程中记录的脑信号分析扩展至基于超声的发音数据。利用从EEG测量的脑信号，我们通过一个全连接深度神经网络预测舌头的超声图像。结果表明，EEG与超声舌图像之间存在微弱但可察觉的关系，即该网络能够区分发音语音和中性舌位。
英文摘要：
Previous initial research has already been carried out to propose speech-based BCI using brain signals (e.g. non-invasive EEG and invasive sEEG / ECoG), but there is a lack of combined methods that investigate non-invasive brain, articulation, and speech signals together and analyze the cognitive processes in the brain, the kinematics of the articulatory movement and the resulting speech signal. In this paper, we describe our multimodal (electroencephalography, ultrasound tongue imaging, and speech) analysis and synthesis experiments, as a feasibility study. We extend the analysis of brain signals recorded during speech production with ultrasound-based articulation data. From the brain signal measured with EEG, we predict ultrasound images of the tongue with a fully connected deep neural network. The results show that there is a weak but noticeable relationship between EEG and ultrasound tongue images, i.e. the network can differentiate articulated speech and neutral tongue position.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
