发表机构
University of Southern California（南加州大学）
机构由 AI 辅助整理，请以论文原文为准。AI 中文总结
本文提出BraVista，一种将多通道脑电图信号编码为结构化图像并利用指令条件化视觉-语言模型进行多任务学习的框架，在睡眠分期、情绪识别等四个数据集上表现优异，验证了结构化视觉表示作为神经信号与基础模型接口的有效性。
AI 中文摘要
学习能够跨认知任务、受试者和记录条件泛化的脑电图（EEG）表征，仍然是脑电图解码中的一个关键挑战。近年来基础模型的进展提高了脑电图解码的性能，但一个基本的开放性问题依然存在：如何有效地将神经信号与这些模型对接，以实现跨数据集的多任务学习。为探究这一问题，我们引入了BraVista，一个视觉-语言框架，它将多通道脑电图信号编码为结构化图像，并通过指令条件化的视觉-语言模型（VLM）实现多任务学习。我们的方法依赖于对通用领域VLM的持续后训练，利用其视觉和语言先验来适应神经信号，而无需单独的大规模脑电图专用预训练阶段。我们在四个数据集上评估了BraVista，涵盖睡眠分期、情绪识别、认知负荷分类和异常脑电图检测，展示了其在这些任务上的强劲性能。进一步的分析表明，脑电图到图像的表示选择对性能至关重要。此外，通过对脑电图信号进行受控扰动，我们观察到在噪声增加时性能逐渐下降，这表明模型依赖于脑电图相关信息而非表面的视觉模式。总之，这些发现确立了结构化视觉表示作为神经信号与通用领域基础模型之间有效且可扩展的接口，用于统一的多任务脑电图解码。
英文摘要
Learning EEG representations that generalize across cognitive tasks, subjects, and recording conditions remains a key challenge in electroencephalography (EEG) decoding. Recent advances in foundation models have improved EEG decoding performance, yet a fundamental open question remains: how to effectively interface neural signals with these models to enable multi-task learning across datasets. To investigate this question, we introduce BraVista, a visual-language framework that encodes multichannel EEG signals as structured images and enables multi-task learning through instruction-conditioned vision-language models (VLMs). Our approach relies on continued post-training of a general-domain VLM, leveraging its visual and linguistic priors to adapt to neural signals without a separate large-scale EEG-specific pretraining stage. We evaluate BraVista on four datasets spanning sleep staging, emotion recognition, cognitive workload classification, and abnormal EEG detection, showing strong performance across these tasks. Further analyses show that the choice of EEG-to-image representation is critical to performance. Moreover, through controlled perturbations of the EEG signal, we observe a gradual performance degradation under increasing noise, suggesting that the model relies on EEG-relevant information rather than superficial visual patterns. Together, these findings establish structured visual representations as an effective and scalable interface between neural signals and general-domain foundation models for unified multi-task EEG decoding.
