arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2107.10552cs.HC
收藏

为在野外建立情感模型提供信息

Privileged Information for Modeling Affect In The Wild

Konstantinos Makantasis, David Melhart, Antonios Liapis, Georgios N. Yannakakis

摘要:

情感计算研究的一个关键挑战是找到可靠地将内置的模型转移到真实世界的方法,即在野外。体外和体内情感应用之间的现有差距主要是由与情感感测相关的限制引起的,包括侵入性、硬件故障、传感器的可用性,以及隐私和安全性。作为对本文中这些限制的回应,我们受到机器学习的最新进展的启发,并引入了特权信息的概念,用于在野外操作情感模型。特权信息的存在使影响模型能够在实验室环境中可用的多种模态中进行训练,并忽略在野外不可用的模态,其建模性能没有显著下降。建议的特权信息框架进行了测试,在游戏唤醒语料库,其中包含生理信号的形式的心率和皮肤电活动,游戏遥测,和像素的镜头从两个不同的游戏,被注释与唤醒痕迹。通过使用所有模态(体外)训练我们的唤醒模型,并仅使用像素来测试模型(体内),我们达到了从融合所有模态用于训练和测试的模型中获得的准确度水平。本文的研究结果为在野外实现情感互动迈出了决定性的一步。

英文摘要:

A key challenge of affective computing research is discovering ways to reliably transfer affect models that are built in the laboratory to real world settings, namely in the wild. The existing gap between in vitro and in vivo affect applications is mainly caused by limitations related to affect sensing including intrusiveness, hardware malfunctions, availability of sensors, but also privacy and security. As a response to these limitations in this paper we are inspired by recent advances in machine learning and introduce the concept of privileged information for operating affect models in the wild. The presence of privileged information enables affect models to be trained across multiple modalities available in a lab setting and ignore modalities that are not available in the wild with no significant drop in their modeling performance. The proposed privileged information framework is tested in a game arousal corpus that contains physiological signals in the form of heart rate and electrodermal activity, game telemetry, and pixels of footage from two dissimilar games that are annotated with arousal traces. By training our arousal models using all modalities (in vitro) and using solely pixels for testing the models (in vivo), we reach levels of accuracy obtained from models that fuse all modalities both for training and testing. The findings of this paper make a decisive step towards realizing affect interaction in the wild.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑