学习儿童不确定性的多模态线索
Learning Multimodal Cues of Children's Uncertainty
AI总结：
本文首次构建了与心理学家合作标注的儿童不确定性非言语线索数据集，并提出一种基于实时视频的多模态机器学习模型来预测不确定性，优于基线transformer模型，为多模态AI协作和手势理解提供支持。
摘要：
理解不确定性在达成共同基础中起着关键作用（Clark et al., 1983）。这对于与用户协作解决问题或引导用户理解具有挑战性概念的多模态AI系统尤为重要。在本工作中，我们首次提出了一个与发展和认知心理学家合作标注的数据集，旨在研究不确定性的非言语线索。随后，我们对数据进行了分析，研究了不确定性的不同作用及其与任务难度和表现之间的关系。最后，我们提出了一个多模态机器学习模型，该模型能够根据参与者的实时视频片段预测不确定性，我们发现该模型优于基线多模态transformer模型。这项工作为人类与人类之间以及人类与AI之间的认知协调研究提供了信息，并对手势理解和生成具有广泛意义。在完成所需的同意书和数据表后，我们的数据和代码的匿名版本将公开提供。
英文摘要：
Understanding uncertainty plays a critical role in achieving common ground (Clark et al.,1983). This is especially important for multimodal AI systems that collaborate with users to solve a problem or guide the user through a challenging concept. In this work, for the first time, we present a dataset annotated in collaboration with developmental and cognitive psychologists for the purpose of studying nonverbal cues of uncertainty. We then present an analysis of the data, studying different roles of uncertainty and its relationship with task difficulty and performance. Lastly, we present a multimodal machine learning model that can predict uncertainty given a real-time video clip of a participant, which we find improves upon a baseline multimodal transformer model. This work informs research on cognitive coordination between human-human and human-AI and has broad implications for gesture understanding and generation. The anonymized version of our data and code will be publicly available upon the completion of the required consent forms and data sheets.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
