一种基于非侵入式记录的多语言脑解码预训练框架
A Pre-trained Framework for Multilingual Brain Decoding Using Non-invasive Recordings
AI总结：
本文提出一种基于预训练多语言模型的统一语义空间框架，实现跨语言、跨被试、跨模态的脑解码，并在159人四语言数据上验证其泛化能力及对语言公平性的促进作用。
摘要：
脑机接口（BCIs）利用脑记录进行语音解码，在临床康复和认知神经科学等领域具有广泛的应用潜力。然而，当前的解码方法仍局限于单语言、单被试和单神经成像模态的设置，限制了其临床适用性和泛化能力。本文提出了一种联合多语言、多被试和多模态的解码框架。该框架将多样的脑记录映射到一个由预训练多语言模型（PMM）定义的统一语义空间中，从而能够跨多种语言、多个被试和多种神经成像模态进行解码。所提出的框架使用来自159名参与者、涵盖四种语言的非侵入式脑记录进行了验证。实验结果表明，该框架在多语言、多被试和多模态设置下表现出强大的泛化能力。更重要的是，所提出的框架能够促进语言公平性，这对BCI应用中代表性不足的语言至关重要。统一的语义空间实现了跨语言映射增强，使该框架能够提升代表性不足语言的解码性能，从而促进语言公平性。总体而言，所提出的框架为脑解码建立了一种新的潜在范式，为BCI的更广泛应用开辟了新路径。
英文摘要：
Brain-computer interfaces (BCIs) with speech decoding from brain recordings have broad application potential in fields such as clinical rehabilitation and cognitive neuroscience. However, current decoding methods remain limited to single-language, single-subject, and single neuroimaging modality settings, restricting their clinical applicability and generalizability. Here we propose a joint multilingual, multi-subject and multimodal decoding framework. It maps diverse brain recordings into a unified semantic space defined by a pre-trained multilingual model (PMM), enabling decoding across multiple languages, multiple subjects and multiple neuroimaging modalities. The proposed framework is validated using non-invasive brain recordings from 159 participants across four languages. Experimental results show that it exhibits strong generalization across multilingual, multi-subject, and multimodal settings. More importantly, the proposed framework can promote linguistic fairness, which is vital for underrepresented languages in BCI applications. The unified semantic space enables cross-lingual mapping enhancement, allowing the framework to boost the decoding performance of underrepresented languages, thereby promoting linguistic fairness. Overall, the proposed framework establishes a new potential paradigm for brain decoding, opening new paths for broader applications of BCI.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
