MMME：一个支持视觉-生理融合的自发多模态微表情数据集
MMME: A Spontaneous Multi-Modal Micro-Expression Dataset Enabling Visual-Physiological Fusion
AI总结：
本研究推出多模态微表情数据集MMME，首次同步采集微表情、脑电及多种外周生理信号，验证了生理信号融合可显著提升微表情识别与定位性能，推动微表情研究从单视觉模态转向多模态融合。
摘要：
微表情（MEs）是揭示个体真实情绪状态的细微、转瞬即逝的非语言线索。由于其在医疗保健、刑事侦查和人机交互等领域具有广阔的应用前景，微表情分析吸引了大量研究兴趣。然而，现有微表情研究局限于单一视觉模态，忽略了其他生理模态传递的丰富情绪信息，导致微表情识别和定位性能远低于实际应用需求。因此，探究微表情视觉特征与生理信号（PS）之间的跨模态关联机制，并开发多模态融合框架，是推动微表情分析发展的关键一步。本研究推出了一个全新的微表情数据集MMME，该数据集首次实现了面部动作信号（微表情）、中枢神经系统信号（EEG，脑电图）以及外周生理信号（PPG光电容积脉搏波、RSP呼吸、SKT皮肤温度、EDA皮肤电活动和ECG心电图）的同步采集。通过突破现有微表情语料库的限制，MMME包含634个微表情、2841个宏表情（MaEs）以及2890组同步多模态生理信号试次，为研究微表情神经机制和开展基于多模态融合的分析奠定了坚实基础。大量实验验证了该数据集的可靠性，并为微表情分析提供了基准，结果表明将微表情与生理信号相结合可显著提升识别和定位性能。据我们所知，就模态多样性而言，MMME是迄今为止最全面的微表情数据集。它为探索微表情的神经机制、揭示视觉-生理协同效应提供了关键的数据支撑，推动微表情研究从单模态视觉分析向多模态融合的范式转变。该数据集将在本文被接收后公开发布。
英文摘要：
Micro-expressions (MEs) are subtle, fleeting nonverbal cues that reveal an individual's genuine emotional state. Their analysis has attracted considerable interest due to its promising applications in fields such as healthcare, criminal investigation, and human-computer interaction. However, existing ME research is limited to single visual modality, overlooking the rich emotional information conveyed by other physiological modalities, resulting in ME recognition and spotting performance far below practical application needs. Therefore, exploring the cross-modal association mechanism between ME visual features and physiological signals (PS), and developing a multimodal fusion framework, represents a pivotal step toward advancing ME analysis. This study introduces a novel ME dataset, MMME, which, for the first time, enables synchronized collection of facial action signals (MEs), central nervous system signals (EEG), and peripheral PS (PPG, RSP, SKT, EDA, and ECG). By overcoming the constraints of existing ME corpora, MMME comprises 634 MEs, 2,841 macro-expressions (MaEs), and 2,890 trials of synchronized multimodal PS, establishing a robust foundation for investigating ME neural mechanisms and conducting multimodal fusion-based analyses. Extensive experiments validate the dataset's reliability and provide benchmarks for ME analysis, demonstrating that integrating MEs with PS significantly enhances recognition and spotting performance. To the best of our knowledge, MMME is the most comprehensive ME dataset to date in terms of modality diversity. It provides critical data support for exploring the neural mechanisms of MEs and uncovering the visual-physiological synergistic effects, driving a paradigm shift in ME research from single-modality visual analysis to multimodal fusion. The dataset will be publicly available upon acceptance of this paper.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
