Mind-to-Face: 通过解码EEG信号实现神经驱动的逼真虚拟形象合成
Mind-to-Face: Neural-Driven Photorealistic Avatar Synthesis via EEG Decoding
AI总结：
Mind-to-Face通过EEG解码实现神经驱动的逼真虚拟形象合成，能够精准捕捉情绪动态并生成高质量的面部表情。
摘要：
当前的表情虚拟形象系统依赖大量视觉线索，在面部被遮挡或情绪保持内部时会失效。我们提出了Mind-to-Face，这是首个能够将非侵入性脑电图（EEG）信号直接解码为高保真面部表情的框架。我们构建了一个双模态记录装置，在引发情绪的刺激下同步获取EEG和多视角面部视频，从而实现对神经到视觉学习的精确监督。我们的模型使用CNN-Transformer编码器将EEG信号映射到密集的3D位置图，能够采样超过65000个顶点，捕捉精细的几何结构和微妙的情绪动态，并通过修改的3D高斯点划管线进行渲染，实现逼真且视图一致的结果。通过广泛的评估，我们证明EEG信号能够可靠地预测动态的、针对个体的面部表情，包括微妙的情绪反应，表明神经信号包含比之前假设更丰富的感情和几何信息。Mind-to-Face建立了一个新的神经驱动虚拟形象范式，使个性化、情绪感知的远程存在和认知交互在沉浸环境中成为可能。
英文摘要：
Current expressive avatar systems rely heavily on visual cues, failing when faces are occluded or when emotions remain internal. We present Mind-to-Face, the first framework that decodes non-invasive electroencephalogram (EEG) signals directly into high-fidelity facial expressions. We build a dual-modality recording setup to obtain synchronized EEG and multi-view facial video during emotion-eliciting stimuli, enabling precise supervision for neural-to-visual learning. Our model uses a CNN-Transformer encoder to map EEG signals into dense 3D position maps, capable of sampling over 65k vertices, capturing fine-scale geometry and subtle emotional dynamics, and renders them through a modified 3D Gaussian Splatting pipeline for photorealistic, view-consistent results. Through extensive evaluation, we show that EEG alone can reliably predict dynamic, subject-specific facial expressions, including subtle emotional responses, demonstrating that neural signals contain far richer affective and geometric information than previously assumed. Mind-to-Face establishes a new paradigm for neural-driven avatars, enabling personalized, emotion-aware telepresence and cognitive interaction in immersive environments.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
