arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新，休息一下吧，周末愉快~~
arXiv 2412.03300cs.ROcs.LG
收藏

触摸与告知：机器人人类情绪与社交手势的多模态解码

Touch and Tell: Multimodal Decoding of Human Emotions and Social Gestures for Robots

Faculty of Engineering and Architecture（工程与建筑学院）；IDLab-AIRO, Ghent University – imec（IDLab-AIRO，根特大学–imec）；Department of Engineering Science, University of Oxford（工程科学系，牛津大学）

Qiaoqiao Ren, Remko Proesmans, Yuanbo Hou, Francis wyffels, Tony Belpaeme

阅读 PDF
arXiv 页面 ↗HTML 原文 ↗

AI总结：

本研究通过在社交机器人上集成压力传感器和麦克风，开发单模态与多模态模型解码触摸和声音传达的人类情绪与社交手势，发现多模态整合显著优于单模态方法，提升了情绪和手势的解码准确率。

摘要：

人类情绪十分复杂，可以通过微妙的触摸手势来传达。以往的研究主要集中在人类如何通过触摸识别情绪，或识别机器人情绪表达的关键特征上。然而，在理解这些情绪和手势如何可靠地通过触摸传达给机器人并使用数据驱动方法进行解释方面仍存在空白。本研究调查了通过触摸和声音表达情绪和手势的一致性与可区分性。为此，我们在社交机器人上集成了定制的压阻式压力传感器以及麦克风。28名参与者首先使用自发的触摸手势向机器人传达十种不同的情绪，然后他们执行了六种预定义的社交触摸手势。我们的发现表明，参与者在情绪和手势表达上具有统计学上的显著一致性。然而，某些情绪表现出较低的内类相关系数值，并且某些具有相似唤醒度或效价水平的情绪在传达上未显示出显著差异。为了研究情感人机触觉交互中的情绪和社交手势解码，我们开发了单模态模型以及整合触觉和听觉特征的多模态模型。在多模态特征上训练的支持向量机（SVM）模型在对十种情绪进行分类时达到了最高准确率，为40%。在手势分类方面，卷积神经网络-长短期记忆网络（CNN-LSTM）达到了90.74%的准确率。我们的结果表明，尽管单模态模型具有解码情绪和触摸手势的潜力，但触摸和声音的多模态整合显著优于单模态方法，增强了对情绪和手势的解码能力。

英文摘要：

Human emotions are complex and can be conveyed through nuanced touch gestures. Previous research has primarily focused on how humans recognize emotions through touch or on identifying key features of emotional expression for robots. However, there is a gap in understanding how reliably these emotions and gestures can be communicated to robots via touch and interpreted using data driven methods. This study investigates the consistency and distinguishability of emotional and gestural expressions through touch and sound. To this end, we integrated a custom piezoresistive pressure sensor as well as a microphone on a social robot. Twenty-eight participants first conveyed ten different emotions to the robot using spontaneous touch gestures, then they performed six predefined social touch gestures. Our findings reveal statistically significant consistency in both emotion and gesture expression among participants. However, some emotions exhibited low intraclass correlation values, and certain emotions with similar levels of arousal or valence did not show significant differences in their conveyance. To investigate emotion and social gesture decoding within affective human-robot tactile interaction, we developed single-modality models and multimodal models integrating tactile and auditory features. A support vector machine (SVM) model trained on multimodal features achieved the highest accuracy for classifying ten emotions, reaching 40 %.For gesture classification, a Convolutional Neural Network- Long Short-Term Memory Network (CNN-LSTM) achieved 90.74 % accuracy. Our results demonstrate that even though the unimodal models have the potential to decode emotions and touch gestures, the multimodal integration of touch and sound significantly outperforms unimodal approaches, enhancing the decoding of both emotions and gestures.

机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。

↑