EEG2Vision：一种基于EEG的多模态框架，用于认知神经科学中的2D视觉重建
EEG2Vision: A Multimodal EEG-Based Framework for 2D Visual Reconstruction in Cognitive Neuroscience
AI总结：
EEG2Vision通过多模态大语言模型和图像扩散模型提升低密度电极配置下的视觉重建质量，验证了低分辨率EEG设备在实时脑-图像应用中的可行性。
摘要：
从非侵入性脑电图（EEG）重建视觉刺激仍然具有挑战性，因为其空间分辨率低且噪声高，特别是在现实中的低密度电极配置下。为此，我们提出了EEG2Vision，一种模块化、端到端的EEG到图像框架，系统评估不同EEG分辨率（128、64、32和24通道）下的重建性能，并通过提示引导的后重建增强机制提升视觉质量。从EEG条件扩散重建开始，增强阶段使用多模态大语言模型提取语义描述，并利用图像到图像扩散模型来细化几何结构和感知一致性，同时保持EEG基础结构。我们的实验表明，随着通道数减少，语义解码准确性显著下降（例如，50-way Top-1 Acc从89%降至38%），而重建质量略有下降（例如，FID从76.77降至80.51）。所提出的增强方法在所有配置中都提高了感知指标，最低通道设置中实现了高达9.71%的IS提升。用户研究证实了增强重建在感知上的明显偏好。所提出的方法显著提高了使用低分辨率EEG设备进行实时脑-图像应用的可行性，可能在实验室之外的环境中解锁此类应用。
英文摘要：
Reconstructing visual stimuli from non-invasive electroencephalography (EEG) remains challenging due to its low spatial resolution and high noise, particularly under realistic low-density electrode configurations. To address this, we present EEG2Vision, a modular, end-to-end EEG-to-image framework that systematically evaluates reconstruction performance across different EEG resolutions (128, 64, 32, and 24 channels) and enhances visual quality through a prompt-guided post-reconstruction boosting mechanism. Starting from EEG-conditioned diffusion reconstruction, the boosting stage uses a multimodal large language model to extract semantic descriptions and leverages image-to-image diffusion to refine geometry and perceptual coherence while preserving EEG-grounded structure. Our experiments show that semantic decoding accuracy degrades significantly with channel reduction (e.g., 50-way Top-1 Acc from 89% to 38%), while reconstruction quality slight decreases (e.g., FID from 76.77 to 80.51). The proposed boosting consistently improves perceptual metrics across all configurations, achieving up to 9.71% IS gains in low-channel settings. A user study confirms the clear perceptual preference for boosted reconstructions. The proposed approach significantly boosts the feasibility of real-time brain-2-image applications using low-resolution EEG devices, potentially unlocking this type of applications outside laboratory settings.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
