基于共享空间对齐的跨主体语义解码用于广义神经表征学习

中文摘要 AI 辅助

在侵入性神经记录中，跨主体泛化仍具有挑战性，因为个体间电极配置、解剖结构和神经信号模式差异很大。为研究这种跨主体变异性，我们提出一个跨主体语义解码框架，将多个主体对语音感知的神经反应对齐到共享潜在空间，并学习从对齐的神经表征到上下文嵌入的映射。具体而言，利用自然语言理解时收集的皮层脑电图数据，用共享响应模型估计共享空间，并训练解码器从投影的神经反应中预测上下文语义嵌入。对于保留主体，估计其到预定义共享空间的特定主体投影，直接应用预训练解码器而无需重新训练。实验结果表明，该框架在各种评估设置下始终优于基线方法，且从源主体到保留主体测试的性能下降减少，表明跨主体泛化得到改善。这些结果表明，将神经活动对齐到共享潜在空间并在语义嵌入空间中解码，通过减少神经反应中的特定主体差异并有效捕获共享刺激相关表征，为改善跨主体泛化提供了有效策略。

英文摘要

Generalizing across subjects remains challenging in invasive neural recordings because electrode configurations, anatomical structures, and neural signal patterns vary substantially across individuals. To investigate such inter-subject variability, we propose a cross-subject semantic decoding framework that aligns neural responses to speech perception from multiple subjects into a shared latent space and learns a mapping from the aligned neural representations to contextual embeddings. More specifically, using electrocorticography data collected during natural language comprehension, we estimate the shared space using the shared response model and train a decoder to predict contextual semantic embeddings from projected neural responses. For a held-out subject, we estimate a subject-specific projection into the predefined shared space, and directly apply the pretrained decoder without any retraining. Experimental results demonstrate that the proposed framework consistently outperforms baseline methods across evaluation settings and exhibits a reduced performance drop from source subject to held-out subject testing, indicating improved cross-subject generalization. These results suggest that aligning neural activity into a shared latent space, while decoding in a semantic embedding space, provides an effective strategy for improving cross-subject generalization by reducing subject-specific differences in neural responses while effectively capturing shared stimulus-related representations.

发表机构

• Korea University（韩国大学）

机构由 AI 辅助整理，请以论文原文为准。