主动倾听者：二元交互中倾听者头部运动响应的连续生成
Active Listener: Continuous Generation of Listener's Head Motion Response in Dyadic Interactions
AI总结：
本文提出一种基于图的端到端跨模态模型，实时从说话者语音生成倾听者连续头部姿态角，在IEMOCAP上实现4.5度低误差和高帧率，验证了其在人机交互中的可部署性。
摘要：
二元口语交互的一个关键组成部分是与语境相关的非语言手势，例如反映倾听者对说话者言语做出响应的头部运动。尽管在生成协同语音手势方面已取得显著进展，但生成倾听者的响应仍然是一个挑战。我们引入了实时生成倾听者响应说话者语音的连续头部运动响应的任务。为此，我们提出了一种基于图的端到端跨模态模型，该模型以对话者的语音音频为输入，并实时直接生成倾听者的头部姿态角（roll、pitch、yaw）。与以往工作不同，我们的方法完全由数据驱动，不需要手动标注，也不会将头部运动过度简化为仅仅点头和摇头。在IEMOCAP数据集上的二元交互会话上进行的广泛评估表明，我们的模型产生了较低的总体误差（4.5度）和较高的帧率，从而表明其可部署于现实世界的人机交互系统中。我们的代码可在 https://github.com/bigzen/Active-Listener 获取。
英文摘要：
A key component of dyadic spoken interactions is the contextually relevant non-verbal gestures, such as head movements that reflect a listener's response to the interlocutor's speech. Although significant progress has been made in the context of generating co-speech gestures, generating listener's response has remained a challenge. We introduce the task of generating continuous head motion response of a listener in response to the speaker's speech in real time. To this end, we propose a graph-based end-to-end crossmodal model that takes interlocutor's speech audio as input and directly generates head pose angles (roll, pitch, yaw) of the listener in real time. Different from previous work, our approach is completely data-driven, does not require manual annotations or oversimplify head motion to merely nods and shakes. Extensive evaluation on the dyadic interaction sessions on the IEMOCAP dataset shows that our model produces a low overall error (4.5 degrees) and a high frame rate, thereby indicating its deployability in real-world human-robot interaction systems. Our code is available at - https://github.com/bigzen/Active-Listener
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
