问题解决过程中被试的多模态观察与解释
Multimodal Observation and Interpretation of Subjects Engaged in Problem Solving
摘要：
在本文中，我们提出了第一个试点实验的结果，在捕获和解释多模态信号的人类专家从事解决具有挑战性的国际象棋问题。我们的目标是调查在何种程度上的眼睛注视，姿势，情绪和其他生理信号的观察可以用来模拟受试者的认知状态，并探讨多传感器模态的整合，以提高可靠性的检测人类显示的意识和情绪。我们观察了国际象棋选手在记录他们的行为时参与难度增加的问题。这种记录可以用来估计参与者对当前情况的意识，并预测有效应对挑战性情况的能力。结果表明，多模态方法比单峰方法更准确。通过结合身体姿势，视觉注意力和情绪，多模态方法可以达到高达93%的准确性时，确定球员的国际象棋专业知识，而单峰方法达到86%。最后，这个实验验证了使用我们的设备作为一个通用的和可重复的工具，参与者从事基于屏幕的互动和/或解决问题的研究。
英文摘要：
In this paper we present the first results of a pilot experiment in the capture and interpretation of multimodal signals of human experts engaged in solving challenging chess problems. Our goal is to investigate the extent to which observations of eye-gaze, posture, emotion and other physiological signals can be used to model the cognitive state of subjects, and to explore the integration of multiple sensor modalities to improve the reliability of detection of human displays of awareness and emotion. We observed chess players engaged in problems of increasing difficulty while recording their behavior. Such recordings can be used to estimate a participant's awareness of the current situation and to predict ability to respond effectively to challenging situations. Results show that a multimodal approach is more accurate than a unimodal one. By combining body posture, visual attention and emotion, the multimodal approach can reach up to 93% of accuracy when determining player's chess expertise while unimodal approach reaches 86%. Finally this experiment validates the use of our equipment as a general and reproducible tool for the study of participants engaged in screen-based interaction and/or problem solving.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
