arXiv 2504.171632025-08-26cs.CV

PhysioSync：受生理同步启发的时序与跨模态对比学习在基于EEG的情绪识别中的应用

PhysioSync: Temporal and Cross-Modal Contrastive Learning Inspired by Physiological Synchronization for EEG-Based Emotion Recognition

School of Instrument Science and Opto-electronics Engineering, Hefei University of Technology（仪器科学与光电工程学院，合肥工业大学）；School of Computer Science and Information Engineering, Hefei University of Technology（计算机科学与信息工程学院，合肥工业大学）

Kai Cui, Jia Li, Yu Liu, Xuesong Zhang, Zhenzhen Hu, Meng Wang

备注：: To appear in IEEE TCSS. The source code is publicly available at https://github.com/MSA-LMC/PhysioSync

阅读 PDF

arXiv 页面 ↗HTML 原文 ↗

AI总结：

针对EEG情绪识别中信号噪声大、个体差异及模态同步被忽视等问题，提出PhysioSync框架，通过跨模态一致性对齐和长短期时序对比学习捕捉模态间/内同步，在DEAP和DREAMER数据集上提升了识别性能。

摘要：

脑电图（EEG）信号为与情绪状态相关的大脑活动提供了一种有前景的非自愿反映，相比面部表情等行为线索具有显著优势。然而，EEG信号通常存在噪声、受伪影影响且个体差异大，使得情绪识别复杂化。尽管多模态方法已使用皮肤电反应（GSR）等外周生理信号（PPS）补充EEG，但它们常忽略模态间的动态同步和一致语义。此外，PPS中不同时间分辨率下情绪波动的时序动态仍未被充分探索。为应对这些挑战，我们提出PhysioSync，一种受生理同步现象启发、利用时序和跨模态对比学习的新型预训练框架。PhysioSync纳入跨模态一致性对齐（CM-CA）以建模EEG与互补PPS间的动态关系，实现跨模态的情绪相关同步；同时引入长短期时序对比学习（LS-TCL），以捕捉模态内不同时间分辨率下的情绪同步。预训练后，跨分辨率和跨模态特征被分层融合并微调，以提升情绪识别效果。在DEAP和DREAMER数据集上的实验表明，PhysioSync在单模态和跨模态条件下均表现出先进性能，突显其对以EEG为中心的情绪识别的有效性。

英文摘要：

Electroencephalography (EEG) signals provide a promising and involuntary reflection of brain activity related to emotional states, offering significant advantages over behavioral cues like facial expressions. However, EEG signals are often noisy, affected by artifacts, and vary across individuals, complicating emotion recognition. While multimodal approaches have used Peripheral Physiological Signals (PPS) like GSR to complement EEG, they often overlook the dynamic synchronization and consistent semantics between the modalities. Additionally, the temporal dynamics of emotional fluctuations across different time resolutions in PPS remain underexplored. To address these challenges, we propose PhysioSync, a novel pre-training framework leveraging temporal and cross-modal contrastive learning, inspired by physiological synchronization phenomena. PhysioSync incorporates Cross-Modal Consistency Alignment (CM-CA) to model dynamic relationships between EEG and complementary PPS, enabling emotion-related synchronizations across modalities. Besides, it introduces Long- and Short-Term Temporal Contrastive Learning (LS-TCL) to capture emotional synchronization at different temporal resolutions within modalities. After pre-training, cross-resolution and cross-modal features are hierarchically fused and fine-tuned to enhance emotion recognition. Experiments on DEAP and DREAMER datasets demonstrate PhysioSync's advanced performance under uni-modal and cross-modal conditions, highlighting its effectiveness for EEG-centered emotion recognition.

机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。