在具有不同一致性的多感官反馈下解码错误相关电位
Decoding Error-Related Potentials under Multisensory Feedback with Varying Congruency
- Korea University（高丽大学）
机构由 AI 辅助整理，请以论文原文为准。
AI总结：
研究在多模态视觉、听觉和触觉反馈及可控感官一致性下的ErrP解码挑战，采用基于多分支EEGNet的架构及辅助监督学习策略，经迷宫观察任务实验，该方法在异质感官条件下分类性能一致且准确度提高，提升了ErrP解码稳定性。
AI中文摘要：
错误相关电位（ErrPs）是人机交互中广泛研究的与错误处理相关的神经信号。在现实环境中，错误感知常在异质多感官反馈下发生，由感官模态和反馈一致性引起的变异性对可靠的ErrP解码构成挑战。特别是不一致的反馈会增加解码难度并降低分类性能。为应对这一挑战，我们研究了在多模态视觉、听觉和触觉反馈以及可控感官一致性下进行稳健ErrP解码的学习策略。我们采用基于多分支EEGNet的架构并辅以辅助监督，以提高跨异质条件的稳健性，而不依赖于明确的特定模态假设。实验使用了具有单模态、双模态和三模态反馈配置的迷宫观察任务。结果表明，所提方法在异质感官条件下实现了一致的分类性能，且与基线EEGNet模型相比准确度有所提高，特别是在多模态反馈下。这些结果表明，适当的架构设计和训练策略可提高异质多感官条件下ErrP解码的稳定性。
英文摘要：
Error-related potentials (ErrPs) are widely studied neural signatures associated with error processing in human-machine interaction. In realistic settings, error perception often occurs under heterogeneous multisensory feedback, where variability induced by sensory modality and feedback congruency poses challenges for reliable ErrP decoding. In particular, incongruent feedback is associated with increased decoding difficulty and reduced classification performance. To address this challenge, we investigate learning strategies for robust ErrP decoding under multimodal visual, auditory, and tactile feedback with controlled sensory congruency. We adopt a multi-branch EEGNet-based architecture with auxiliary supervision to improve robustness across heterogeneous conditions, without relying on explicit modality-specific assumptions. Experiments were conducted using a maze-observation task with unimodal, bimodal, and trimodal feedback configurations. Across subjects, the proposed approach achieved consistent classification performance across heterogeneous sensory conditions and showed improved accuracy compared to baseline EEGNet models, particularly under multimodal feedback. These results suggest that appropriate architectural design and training strategies can improve the stability of ErrP decoding under heterogeneous multisensory conditions.
