基于多模态可穿戴设备的嗅觉诱导唤醒-效价维度情感识别

中文摘要 AI 辅助

嗅觉对情感调节至关重要，因为它是一种非侵入性、认知负荷低的通路，可直接作用于大脑情感回路，实现无干扰的情感调节，这一特性对推进日常及注意力关键场景下的实用情感计算具有重要意义。然而当前嗅觉情感研究存在两个关键局限：一是过度强调效价维度而忽视唤醒维度；二是缺乏同步捕捉嗅觉刺激下中枢与外周生理反应的多模态数据集。为解决这些问题，本研究构建了包含111名受试者的大规模多模态嗅觉情感数据集，其中气味在二维唤醒-效价空间中标注，同步记录了脑电图（EEG）、心电图（ECG）和光体积描记图（PPG）信号。不过，多模态信号存在非平稳性、延迟差异及跨模态异质性等挑战，因此本研究提出时空频率混合融合网络（STF-HFNet），该网络整合了三个核心模块：频率聚合处理模块学习自适应频率聚合以建模非平稳动态；互导注意力模块实现无同步先验的跨模态时间对齐的双向校准；混合协同融合模块结合空间与通道注意力机制，增强跨模态互补性同时抑制冗余信息。大量实验表明，STF-HFNet在AMIGOS数据集上达到88.34%的最先进（SOTA）识别准确率，在自建数据集上达到92.40%的准确率，分别比现有SOTA方法高出8.27%和5.07%。

英文摘要

Olfaction is important for emotion regulation because it acts as a non-intrusive and cognitively lightweight pathway that directly engages the brain s affective circuitry and achieves unobtrusive emotional modulation. This trait is essential for advancing practical affective computing in daily and attention-critical scenarios. However, current olfactory emotion research has two key limitations. First, it overemphasises the valence dimension while neglecting arousal. Second, it lacks multimodal datasets that synchronously capture central and peripheral physiological responses to olfactory stimuli. To address these issues, we construct a large-scale multimodal olfactory emotion dataset based on 111 subjects, in which odors are labeled in the 2D arousal-valence space and electroencephalogram (EEG), electrocardiogram (ECG), and photoplethysmography (PPG) signals synchronously recorded. Nevertheless, multimodal signals present challenges such as non-stationarity, differences in latency, and cross-modal heterogeneity. Thus, we propose a spatiotemporal-frequency hybrid fusion network (STF-HFNet), which integrates three core modules. Frequency aggregation processing learns adaptive frequency aggregation in order to model non-stationary dynamics. Reciprocal guided attention enables reciprocal bidirectional calibration for cross-modal temporal alignment without synchronisation priors. Hybrid collaborative fusion combines spatial and channel attention mechanisms to enhance cross-modal complementarity while suppressing redundant information. Extensive experiments show that STF-HFNet achieves state-of-the-art (SOTA) recognition accuracies of 88.34% on the AMIGOS dataset and 92.40% on our self-constructed dataset, and outperform the SOTA methods by 8.27% and 5.07%, respectively.

发表机构

• Tianjin University（天津大学）

机构由 AI 辅助整理，请以论文原文为准。