IDEAL：一种用于脑电图-眼动情绪识别的多模态域自适应框架

中文摘要 AI 辅助

脑电图（EEG）情绪识别是人机交互的关键接口，然而个体间的生理差异使得融合异质生理信号（如脑电图和眼动）这一本已具有挑战性的任务更加复杂。然而，大多数流行的域自适应范式针对单模态场景设计，无法解决多模态信号的异质性。此外，它们主要依赖特征级对齐，忽视了基本的数据级差异，这在激进的自适应过程中可能损害细粒度的判别信息。为弥合这些耦合的差距，我们提出了基于实例的域扩展与对抗学习（IDEAL），这是一个统一框架，将实例级课程扩展与特征级分层对抗对齐协同起来。IDEAL首先引入多模型协作筛选机制，通过数量-质量均衡策略传播高置信度目标样本，以在数据层面明确弥合分布差距。我们提供了理论分析，证明这种实例扩展策略严格收紧目标风险的上界。随后，一个增强有角度对比约束的分层对抗网络，从低级统计到高级语义逐步对齐表征，同时保持类别可分性。在四个基准数据集上的大量实验表明，IDEAL显著优于最先进的方法。为促进可复现性和未来研究，我们的源代码已公开于该https URL。

英文摘要

Electroencephalography (EEG) emotion recognition serves as a pivotal interface for human-computer interaction, yet the physiological variability across individuals complicates the already challenging task of fusing heterogeneous physiological signals (e.g., EEG and eye movements). However, most prevalent domain adaptation paradigms are tailored for unimodal scenarios, failing to address the heterogeneity of multimodal signals. Furthermore, they predominantly rely on feature-level alignment, overlooking the fundamental data-level discrepancy, which risks compromising fine-grained discriminative information during aggressive adaptation. To bridge these coupled gaps, we propose Instance-based Domain Expansion and Adversarial Learning (IDEAL), a unified framework that synergizes instance-level curriculum expansion with feature-level hierarchical adversarial alignment. IDEAL first introduces a multi-model collaborative screening mechanism, which propagates high-confidence target samples to explicitly bridge the distributional gap at the data level via a quantity-quality equilibrium strategy. We provide a theoretical analysis that this instance expansion strategy strictly tightens the upper bound of the target risk. Subsequently, a hierarchical adversarial network, augmented with the angular-contrastive constraints, progressively aligns representations from low-level statistics to high-level semantics while preserving class separability. Extensive experiments on four benchmark datasets demonstrate that IDEAL significantly outperforms state-of-the-art methods. To facilitate reproducibility and future research, our source code is publicly available at https://github.com/WY-BCI-Club/IDEAL.

发表机构

• South China University of Technology（华南理工大学）
• Institute of Automation, Chinese Academy of Sciences（中国科学院自动化研究所）
• University of California at San Diego（加州大学圣地亚哥分校）

机构由 AI 辅助整理，请以论文原文为准。