评估应用婴儿情感识别中的时间模式
Evaluating Temporal Patterns in Applied Infant Affect Recognition
AI总结:
针对现有婴儿情感识别评估未考虑遮挡、状态切换下性能变化的问题,研究者结合面部与身体特征训练分类器,发现多模态模型整体更优,且状态转换时性能最弱,凸显了时间维度与缺失数据下性能评估的重要性。
摘要:
智能体必须持续监测交互伙伴的情感状态,才能理解并参与社交互动。然而,现有的情感识别评估方法并未考虑遮挡发生期间或情感状态切换过程中分类性能可能出现的变化。本文针对婴儿-机器人交互场景下情感分类性能的时间模式展开研究,该场景中婴儿的情感状态会影响其参与腿部运动治疗活动的能力。为提升对视频录制中面部遮挡的鲁棒性,我们同时利用面部特征和身体特征训练了婴儿情感识别分类器。随后,我们对性能最优的模型进行了深入分析,以评估当模型遇到数据缺失及婴儿情感变化时,性能随时间的变化情况。在特征提取置信度较高的时间窗口内,仅基于面部特征训练的单模态模型取得了与同时基于面部和身体特征训练的多模态模型相同的最优性能。但在整个数据集上评估时,多模态模型的表现优于单模态模型。此外,模型在预测情感状态转换时性能最弱,在对同一情感状态进行多次预测后性能会有所提升。这些发现凸显了在婴儿连续情感识别中纳入身体特征的优势。我们的工作强调,在将情感识别应用于社交互动时,评估模型性能随时间的变化以及存在缺失数据时的性能变化具有重要意义。
英文摘要:
Agents must monitor their partners' affective states continuously in order to understand and engage in social interactions. However, methods for evaluating affect recognition do not account for changes in classification performance that may occur during occlusions or transitions between affective states. This paper addresses temporal patterns in affect classification performance in the context of an infant-robot interaction, where infants' affective states contribute to their ability to participate in a therapeutic leg movement activity. To support robustness to facial occlusions in video recordings, we trained infant affect recognition classifiers using both facial and body features. Next, we conducted an in-depth analysis of our best-performing models to evaluate how performance changed over time as the models encountered missing data and changing infant affect. During time windows when features were extracted with high confidence, a unimodal model trained on facial features achieved the same optimal performance as multimodal models trained on both facial and body features. However, multimodal models outperformed unimodal models when evaluated on the entire dataset. Additionally, model performance was weakest when predicting an affective state transition and improved after multiple predictions of the same affective state. These findings emphasize the benefits of incorporating body features in continuous affect recognition for infants. Our work highlights the importance of evaluating variability in model performance both over time and in the presence of missing data when applying affect recognition to social interactions.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。