当我和你说话时看着我：一个视频数据集，使语音助理能够识别错误
Look at Me When I Talk to You: A Video Dataset to Enable Voice Assistants to Recognize Errors
摘要：
与语音助手互动的人经常会因为语音助手的频繁错误和无法响应反向通道提示而感到沮丧。我们介绍了一个开源视频数据集，其中包含21名参与者与语音助手的交互，并探索了使用该数据集实现自动错误识别以告知自我修复的可能性。该数据集包括从智能扬声器的角度与语音助手进行自由形式交互期间参与者面部的剪辑和标记视频。为了验证我们的数据集，我们模拟了一个机器学习分类器，要求众包工作人员通过观看参与者反应的无声视频片段来识别语音助理错误。我们发现的趋势表明，仅从参与者的面部反应就可以确定语音助手的性能。这项工作假设引发了交互式响应的数据集，作为改善各种应用中语音助手修复错误识别的关键一步。
英文摘要：
People interacting with voice assistants are often frustrated by voice assistants' frequent errors and inability to respond to backchannel cues. We introduce an open-source video dataset of 21 participants' interactions with a voice assistant, and explore the possibility of using this dataset to enable automatic error recognition to inform self-repair. The dataset includes clipped and labeled videos of participants' faces during free-form interactions with the voice assistant from the smart speaker's perspective. To validate our dataset, we emulated a machine learning classifier by asking crowdsourced workers to recognize voice assistant errors from watching soundless video clips of participants' reactions. We found trends suggesting it is possible to determine the voice assistant's performance from a participant's facial reaction alone. This work posits elicited datasets of interactive responses as a key step towards improving error recognition for repair for voice assistants in a wide variety of applications.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
