情绪理解是一个涉及多个成分的复杂过程。识别情绪的能力不仅带来了新的情境感知方法,还通过感知和表达情绪提升了系统交互的有效性。尽管离散模型和维度模型受到关注,但神经科学证据支持情绪是复杂且多面的。与这些发现高度契合的一个框架是成分过程模型(Component Process Model,CPM),该理论通过五个相互关联的成分来考察情绪的复杂性:评估、表达、动机、生理和感受。然而,CPM 与离散情绪之间的关系尚未得到充分探索。因此,为了更好地理解情绪的潜在过程,我们采用交互式虚拟现实(VR)游戏实施了一种数据驱动方法,并从 39 名参与者处收集了多模态测量数据(自我报告、生理信号和面部信号)。我们使用机器学习(ML)方法识别每个成分对情绪区分的独特贡献。结果揭示了不同成分在情绪区分中的作用,其中包含全部成分的模型表现出最显著的贡献。此外,我们发现,在我们的数据集中至少需要五个维度才能表征情绪的变化。这些发现还对在情绪研究中使用 VR 环境具有启示,并凸显了在此类环境中生理信号在情绪识别中的作用。
英文摘要:
Emotion understanding is a complex process that involves multiple components. The ability to recognise emotions not only leads to new context awareness methods but also enhances system interaction's effectiveness by perceiving and expressing emotions. Despite the attention to discrete and dimensional models, neuroscientific evidence supports those emotions as being complex and multi-faceted. One framework that resonated well with such findings is the Component Process Model (CPM), a theory that considers the complexity of emotions with five interconnected components: appraisal, expression, motivation, physiology and feeling. However, the relationship between CPM and discrete emotions has not yet been fully explored. Therefore, to better understand emotions underlying processes, we operationalised a data-driven approach using interactive Virtual Reality (VR) games and collected multimodal measures (self-reports, physiological and facial signals) from 39 participants. We used Machine Learning (ML) methods to identify the unique contributions of each component to emotion differentiation. Our results showed the role of different components in emotion differentiation, with the model including all components demonstrating the most significant contribution. Moreover, we found that at least five dimensions are needed to represent the variation of emotions in our dataset. These findings also have implications for using VR environments in emotion research and highlight the role of physiological signals in emotion recognition within such environments.
EMooly: Supporting Autistic Children in Collaborative Social-Emotional Learning with Caregiver Participation through Interactive AI-infused and AR Activities
University of Waterloo(滑铁卢大学);Southern University of Science and Technology(南方科技大学);Hong Kong University of Science and Technology (Guangzhou)(香港科技大学(广州));National Research Council(国家研究委员会)
CogIntAc: Modeling the Relationships between Intention, Emotion and Action in Interactive Process from Cognitive Perspective
Institute of Information Engineering, Chinese Academy of Sciences(中国科学院信息工程研究所);School of Cyber Security, University of Chinese Academy of Sciences(中国科学院大学网络空间安全学院)
Understanding the Effects of Interaction on Emotional Experiences in VR
The University of Sydney(悉尼大学);Nanjing University of Information Science and Technology(南京信息工程大学);TU Dortmund University(多特蒙德工业大学);Research Center Trustworthy Data Science and Security(可信数据科学与安全研究中心);University of New South Wales(新南威尔士大学)
Teaching AI to Feel: A Collaborative, Full-Body Exploration of Emotive Communication
Institute of Neurosciences of the University of Barcelona Spain;Artificial Intelligence Research Institute (IIIA-CSIC) Barcelona Spain;Massachusetts Institute of Technology Cambridge USA;ESPRONCEDA Institute of Art \& Culture Barcelona Spain;Institute of Neurosciences of the University of;Artificial Intelligence Research Institute (IIIA-CSIC);Massachusetts Institute of Technology;ESPRONCEDA Institute of Art \& Culture