面对面场景中的情感建模：眼动追踪、人格特质与时间动态的交互作用
Modelling Emotions in Face-to-Face Setting: The Interplay of Eye-Tracking, Personality, and Temporal Dynamics
AI总结：
本研究提出一种整合眼动追踪、时间动态与人格特质的神经网络模型，显著提升了面对面动态场景下感知情绪与体验情绪的识别准确率，为情感计算与人机交互系统设计提供了重要启示。
摘要：
准确的情感识别对于实现细腻且引人入胜的人机交互至关重要，但仍然难以实现，特别是在动态的、类似对话的场景中。在本研究中，我们展示了如何整合眼动追踪数据、时间动态和人格特质，从而显著提升对感知情绪和体验情绪的检测能力。73名参与者观看了来自CREMA-D数据集的包含语音的短视频，同时记录其眼动追踪信号（瞳孔大小、注视模式）、大五人格评估结果以及自我报告的情绪状态。我们的神经网络模型结合了这些多样化的输入，包括用于提供上下文线索的刺激情绪标签，与现有最先进方法相比，取得了显著的性能提升。具体而言，感知效价预测的宏F1分数达到了0.76，并且融合了人格特质和刺激信息的模型在体验情绪准确率上展现出显著提升。这些结果凸显了统合生理、个体和情境因素以解决情感表达的主观性与复杂性的优势。除了验证用户特定数据在捕捉微妙内部状态中的作用外，我们的研究结果还为未来情感计算和人类-智能体系统的设计提供了启示，为在真实世界交互中实现更具适应性和跨个体的情感智能铺平了道路。
英文摘要：
Accurate emotion recognition is pivotal for nuanced and engaging human-computer interactions, yet remains difficult to achieve, especially in dynamic, conversation-like settings. In this study, we showcase how integrating eye-tracking data, temporal dynamics, and personality traits can substantially enhance the detection of both perceived and felt emotions. Seventy-three participants viewed short, speech-containing videos from the CREMA-D dataset, while being recorded for eye-tracking signals (pupil size, fixation patterns), Big Five personality assessments, and self-reported emotional states. Our neural network models combined these diverse inputs including stimulus emotion labels for contextual cues and yielded marked performance gains compared to the state-of-the-art. Specifically, perceived valence predictions reached a macro F1-score of 0.76, and models incorporating personality traits and stimulus information demonstrated significant improvements in felt emotion accuracy. These results highlight the benefit of unifying physiological, individual and contextual factors to address the subjectivity and complexity of emotional expression. Beyond validating the role of user-specific data in capturing subtle internal states, our findings inform the design of future affective computing and human-agent systems, paving the way for more adaptive and cross-individual emotional intelligence in real-world interactions.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
