MindAU: 基于双流流形对齐的脑电条件面部动作单元编辑
MindAU: EEG-Conditioned Facial Action Unit Editing via Dual-Stream Manifold Alignment
AI总结：
提出MindAU框架，通过双流流形对齐将EEG特征与AU文本语义和视觉位移对齐，实现高保真身份保持的面部动作单元编辑。
摘要：
最近的脑解码研究在从神经信号重建外部感知视觉内容方面取得了实质性进展。然而，使用脑电图（EEG）记录来指导面部表情编辑仍然基本未被探索，并提出了一个独特的挑战：它需要从嘈杂的EEG信号中识别与面部动作相关的模式，并将其定位到局部、保持身份的表情编辑中，而不是恢复受试者看到的内容。在本文中，我们研究了EEG条件的面部图像编辑，用于细粒度的面部动作单元（AU）控制，并提出了MindAU，一个用于从EEG信号控制面部AU编辑的统一框架。MindAU首先通过时间掩码重建和AU分类监督学习噪声鲁棒且AU判别性的EEG表示。然后，它通过双流流形对齐弥合模态差距，在Qwen2.5-VL的多模态空间中将EEG特征与AU级文本语义和身份减少的视觉位移轨迹对齐。最后，MindAU将EEG感知的多模态旋转位置嵌入、地标引导的参考掩码和AU感知区域监督集成到多模态扩散编辑器中，用于高保真身份保持编辑。我们还引入了E-CAFE，一个精心策划的基准，用于EEG条件动作单元面部编辑，包含配对的EEG-面部编辑样本和标准化评估协议。大量实验证明了MindAU的有效性，并表明其作为未来面部神经肌肉障碍患者辅助表达技术的潜力。
英文摘要：
Recent brain decoding studies have made substantial progress in reconstructing externally perceived visual content from neural signals. However, using electroencephalography (EEG) recordings to guide facial expression editing remains largely unexplored and poses a distinct challenge: rather than recovering what a subject sees, it requires identifying facial-action related patterns from noisy EEG signals and grounding them in localized, identity-preserving expression edits. In this paper, we investigate EEG-conditioned facial image editing for fine-grained facial action unit (AU) control and propose MindAU, a unified framework for controlling facial AU edits from EEG signals. MindAU first learns noise-robust and AU-discriminative EEG representations through temporal masked reconstruction and AU classification supervision. It then bridges the modality gap via Dual-Stream Manifold Alignment, aligning EEG features with AU-level text semantics and identity-reduced visual displacement trajectories in the multimodal space of Qwen2.5-VL. Finally, MindAU incorporates EEG-aware Multimodal Rotary Positional Embeddings, landmark-guided reference masking, and AU-aware region supervision into a multimodal diffusion-based editor for high-fidelity identity-preserving editing. We also introduce E-CAFE, a curated benchmark for EEG-Conditioned Action-Unit Facial Editing with paired EEG-face editing samples and standardized evaluation protocols. Extensive experiments demonstrate the effectiveness of MindAU and suggest its potential as a step towards future assistive expression technologies for individuals with facial neuromuscular disorders.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
