DynaMind：通过对齐时间动态与多模态语义到引导扩散模型，从EEG重建动态视觉场景
DynaMind: Reconstructing Dynamic Visual Scenes from EEG by Aligning Temporal Dynamics and Multimodal Semantics to Guided Diffusion
AI总结：
针对EEG重建动态视觉场景的三大难点，提出含三个核心模块的DynaMind框架，在SEED-DV数据集上实现SOTA，大幅提升重建准确率与视觉质量，推动脑解码领域进展。
摘要：
从脑电图（EEG）信号重建动态视觉场景仍是脑解码领域的核心挑战，受限于EEG的低空间分辨率、神经记录与视频动态之间的时间不匹配，以及脑活动中语义信息利用不足的问题。因此，现有方法往往无法充分还原感知视觉刺激的动态连贯性与复杂语义语境。为克服这些局限，我们提出DynaMind——一种新型框架，通过三个核心模块联合建模神经动态与语义特征以重建视频：区域感知语义映射器（Regional-aware Semantic Mapper, RSM）、时间感知动态对齐器（Temporal-aware Dynamic Aligner, TDA），以及双引导视频重建器（Dual-Guidance Video Reconstructor, DGVR）。RSM首先利用区域感知编码器，从不同脑区的EEG信号中提取多模态语义特征，并将其聚合为统一的扩散先验。同时，TDA生成动态潜序列（即蓝图），以增强特征表示与原始神经记录之间的时间一致性。随后，在语义扩散先验的引导下，DGVR将时间感知蓝图转换为高保真视频重建结果。在SEED-DV数据集上，DynaMind创下新的 state-of-the-art（SOTA，当前最优水平），将基于视频和基于帧的重建准确率分别提升了12.5和10.3个百分点。该方法还在像素级质量上实现了飞跃，展现出卓越的视觉保真度与时间连贯性，SSIM提升9.4%，FVMD降低19.7%。这标志着一项关键进展，填补了神经动态与高保真视觉语义之间的鸿沟。
英文摘要：
Reconstruction dynamic visual scenes from electroencephalography (EEG) signals remains a primary challenge in brain decoding, limited by the low spatial resolution of EEG, a temporal mismatch between neural recordings and video dynamics, and the insufficient use of semantic information within brain activity. Therefore, existing methods often inadequately resolve both the dynamic coherence and the complex semantic context of the perceived visual stimuli. To overcome these limitations, we introduce DynaMind, a novel framework that reconstructs video by jointly modeling neural dynamics and semantic features via three core modules: a Regional-aware Semantic Mapper (RSM), a Temporal-aware Dynamic Aligner (TDA), and a Dual-Guidance Video Reconstructor (DGVR). The RSM first utilizes a regional-aware encoder to extract multimodal semantic features from EEG signals across distinct brain regions, aggregating them into a unified diffusion prior. In the mean time, the TDA generates a dynamic latent sequence, or blueprint, to enforce temporal consistency between the feature representations and the original neural recordings. Together, guided by the semantic diffusion prior, the DGVR translates the temporal-aware blueprint into a high-fidelity video reconstruction. On the SEED-DV dataset, DynaMind sets a new state-of-the-art (SOTA), boosting reconstructed video accuracies (video- and frame-based) by 12.5 and 10.3 percentage points, respectively. It also achieves a leap in pixel-level quality, showing exceptional visual fidelity and temporal coherence with a 9.4% SSIM improvement and a 19.7% FVMD reduction. This marks a critical advancement, bridging the gap between neural dynamics and high-fidelity visual semantics.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
