发表机构
Fudan University; Westlake University; Zhejiang University; Nanyang Technological University; Xmov（复旦大学; 西湖大学; 浙江大学; 南洋理工大学; 未提及通用中文名，可音译为艾莫夫）
机构由 AI 辅助整理，请以论文原文为准。AI 中文总结
该研究提出fMRI-Face数据集及fMRI2Face框架，用于从大脑活动重建动态人脸。框架从大脑活动中获取两种互补神经控制，经整合实现高保真面部视频重建，实验显示其性能优于基线，为动态面部感知研究提供了新平台和基准。
AI 中文摘要
从大脑活动重建动态人脸为研究大脑如何感知身份、表情和面部运动提供了有力途径。然而，基于功能磁共振成像（fMRI）的面部解码进展受限，缺乏高分辨率神经数据集，且方法难以恢复特定身份外观和时变面部动态。我们提出了fMRI-Face，这是首个与1920×1080分辨率的可控全高清数字人脸视频配对的fMRI数据集。在此基础上，我们提出了fMRI2Face，一种用于从fMRI信号重建面部视频的几何引导神经视频解码框架。实验表明，fMRI2Face在重建保真度、身份保留、面部几何形状和运动一致性方面优于代表性神经解码基线。fMRI-Face和fMRI2Face为研究动态面部感知建立了一个可控平台，并为基于fMRI的数字人类重建提供了新基准。
英文摘要
Reconstructing dynamic human faces from brain activity provides a powerful way to study how the mind perceives identity, expression, and facial motion. However, progress in fMRI-based face decoding has been limited by scarce controlled, high-resolution neural datasets and by methods that struggle to recover both identity-specific appearance and time-varying facial dynamics. We present fMRI-Face, the first fMRI dataset paired with controllable full-HD digital human facial videos rendered at 1920$\times$1080 resolution. During scanning, participants watched photorealistic, background-free facial videos with controlled identity, expression, and head pose, while fMRI activity was recorded. The resulting dataset contains 62,856 paired fMRI-video samples, providing a structured resource for studying dynamic face perception and reconstruction. Building on this dataset, we propose fMRI2Face, a geometry-guided neural video decoding framework for reconstructing facial videos from fMRI signals. fMRI2Face derives two complementary neural controls from brain activity: Brain-derived Appearance Context, which captures global identity-related visual attributes, and Morphable 3D Facial Control, which provides explicit geometry-aware guidance for pose, expression, and non-rigid facial dynamics. These controls are integrated through Neural-Controlled Video Diffusion with auxiliary latent completion, enabling high-fidelity facial video reconstruction directly from brain activity. Experiments show that fMRI2Face consistently improves reconstruction fidelity, identity preservation, facial geometry, and motion consistency over representative neural decoding baselines. Together, fMRI-Face and fMRI2Face establish a controlled platform for studying dynamic face perception and provide a new benchmark for fMRI-based digital human reconstruction.
