基于sEEG的句子检索编码：脑-语言对齐的对比学习方法
sEEG-based Encoding for Sentence Retrieval: A Contrastive Learning Approach to Brain-Language Alignment
AI总结：
本文提出SSENSE对比学习框架，将sEEG信号映射到CLIP句子空间，实现基于脑活动的句子检索，验证了通用语言表征作为神经解码先验的可行性。
摘要：
通过有意义的潜在表征来解释神经活动，是神经科学与人工智能交叉领域一个复杂且不断演变的挑战。我们研究了多模态基础模型在将侵入性脑记录与自然语言对齐方面的潜力。我们提出了SSENSE，一个对比学习框架，将单被试立体脑电图（sEEG）信号投影到冻结的CLIP模型的句子嵌入空间中，从而实现直接从脑活动进行句子级检索。SSENSE使用InfoNCE损失在sEEG的频谱表征上训练神经编码器，而不微调文本编码器。我们在一个自然电影观看数据集上，对时间对齐的sEEG和口语转录进行了方法评估。尽管数据有限，SSENSE取得了有前景的结果，表明通用语言表征可以作为神经解码的有效先验。
英文摘要：
Interpreting neural activity through meaningful latent representations remains a complex and evolving challenge at the intersection of neuroscience and artificial intelligence. We investigate the potential of multimodal foundation models to align invasive brain recordings with natural language. We present SSENSE, a contrastive learning framework that projects single-subject stereo-electroencephalography (sEEG) signals into the sentence embedding space of a frozen CLIP model, enabling sentence-level retrieval directly from brain activity. SSENSE trains a neural encoder on spectral representations of sEEG using InfoNCE loss, without fine-tuning the text encoder. We evaluate our method on time-aligned sEEG and spoken transcripts from a naturalistic movie-watching dataset. Despite limited data, SSENSE achieves promising results, demonstrating that general-purpose language representations can serve as effective priors for neural decoding.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
