Ain't Misbehavin'——使用LLMs在桌面机器人Haru的对话中生成富有表现力的机器人行为
Ain't Misbehavin' -- Using LLMs to Generate Expressive Robot Behavior in Conversations with the Tabletop Robot Haru
AI总结：
本文提出将LLMs集成到桌面机器人Haru中构建全自动对话系统，结合TTS引擎、物理动作库与情感识别模型生成富有表现力的行为，试点研究表明反馈积极，并揭示了LLM幻觉等错误对对话的潜在影响。
摘要：
社交机器人旨在通过引人入胜的对话与人类建立长期联系。然而，依赖预设脚本交互的传统对话方法在维持引人入胜的对话方面常常存在不足。本文通过将大语言模型（LLMs）集成到社交机器人中来解决这一局限性，以实现更动态和富有表现力的对话。我们引入了一个全自动对话系统，利用LLMs生成符合机器人个性的富有表现力的机器人回复。我们通过两种模态结合机器人行为：1）支持多种表达风格的文本转语音（TTS）引擎，以及2）机器人的物理动作库。我们开发了一个定制的、最先进的情感识别模型来动态选择机器人的语调，并利用LLM输出中的表情符号作为生成机器人动作的线索。我们的系统演示可在此处获取。为了阐明设计和实现问题，我们进行了一项试点研究，志愿者使用我们提出的系统与社交机器人聊天，我们分析了他们的反馈，并对聊天记录进行了严格的错误分析。反馈绝大多数是积极的，参与者对机器人的同理心、有用性、自然性和娱乐性发表了评论。大多数负面反馈是由于自动语音识别（ASR）错误造成的，这些错误对对话的影响有限。然而，我们观察到一小类错误，例如LLM重复自身或产生虚构信息及人类回复的幻觉，这些错误有可能使对话偏离正轨，引发了LLM应用的重要问题。
英文摘要：
Social robots aim to establish long-term bonds with humans through engaging conversation. However, traditional conversational approaches, reliant on scripted interactions, often fall short in maintaining engaging conversations. This paper addresses this limitation by integrating large language models (LLMs) into social robots to achieve more dynamic and expressive conversations. We introduce a fully-automated conversation system that leverages LLMs to generate robot responses with expressive behaviors, congruent with the robot's personality. We incorporate robot behavior with two modalities: 1) a text-to-speech (TTS) engine capable of various delivery styles, and 2) a library of physical actions for the robot. We develop a custom, state-of-the-art emotion recognition model to dynamically select the robot's tone of voice and utilize emojis from LLM output as cues for generating robot actions. A demo of our system is available here. To illuminate design and implementation issues, we conduct a pilot study where volunteers chat with a social robot using our proposed system, and we analyze their feedback, conducting a rigorous error analysis of chat transcripts. Feedback was overwhelmingly positive, with participants commenting on the robot's empathy, helpfulness, naturalness, and entertainment. Most negative feedback was due to automatic speech recognition (ASR) errors which had limited impact on conversations. However, we observed a small class of errors, such as the LLM repeating itself or hallucinating fictitious information and human responses, that have the potential to derail conversations, raising important issues for LLM application.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
