arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2112.14928cs.HC
收藏

基于文本的人与人对话非语言注释系统的用户实证研究

An empirical user-study of text-based nonverbal annotation systems for human-human conversations

Joshua Y. Kim, Kalina Yacef

摘要:

随着在线人与人对话数量的大幅增加以及多模态转录的实用性,对自动多模态转录系统的需求不断增加,以帮助我们更好地理解对话。在本文中,我们评估了三种方法来执行多模态转录。它们是(1)Jefferson --语言学界广泛使用的现有手动系统,(2)MONAH --一个旨在使多模态转录可访问和自动化的系统,(3)MONAH+ --一个基于MONAH的可视化机器注意力的系统。基于104名参与者的回答,我们发现:(1)所有基于文本的方法都显着减少了人类用户的信息量,(2)MONAH被发现比Jefferson更有用,(3)Jefferson的相对优势在于时间顺序(pace / delay)和语言学(音调/音量)注释,而MONAH的相对优势在于身势语(身体语言)注释,(4)基于机器注意力放大单词的字体大小会使人类用户感到困惑,因为音量很大。这些结果为研究人员设计多模态注释系统提供了考虑,这些系统可以为想要全自动或人工增强会话分析系统的大众提供多模态注释系统。

英文摘要:

the substantial increase in the number of online human-human conversations and the usefulness of multimodal transcripts, there is a rising need for automated multimodal transcription systems to help us better understand the conversations. In this paper, we evaluated three methods to perform multimodal transcription. They were (1) Jefferson -- an existing manual system used widely by the linguistics community, (2) MONAH -- a system that aimed to make multimodal transcripts accessible and automated, (3) MONAH+ -- a system that builds on MONAH that visualizes machine attention. Based on 104 participants responses, we found that (1) all text-based methods significantly reduced the amount of information for the human users, (2) MONAH was found to be more usable than Jefferson, (3) Jefferson's relative strength was in chronemics (pace / delay) and paralinguistics (pitch / volume) annotations, whilst MONAH's relative strength was in kinesics (body language) annotations, (4) enlarging words' font-size based on machine attention was confusing human users as loudness. These results pose considerations for researchers designing a multimodal annotation system for the masses who would like a fully-automated or human-augmented conversational analysis system.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑