当机器人变得健谈：多模态人机对话与协作的具身化
When Robots Get Chatty: Grounding Multimodal Human-Robot Conversation and Collaboration
AI总结：
本文提出一种模块化方法，将大语言模型与物理机器人感知及多种深度学习模型集成，实现开放式人机对话与协作，并验证了其在自然社交交互中的潜力。
摘要：
我们研究了使用大语言模型（LLMs）为神经机器人智能体赋予类人的社交与认知能力，以实现开放式人机对话与协作。我们提出了一种模块化且可扩展的方法，将大语言模型与物理机器人的感官感知和能力进行具身化连接，并以系统集成的方式在整个架构中整合了多个深度学习模型。这些集成模型涵盖语音识别、语音生成、开放词汇目标检测、人体姿态估计和手势检测等多种功能，大语言模型作为核心的基于文本的协调单元。定性和定量结果展示了大语言模型在以自然和社交方式为机器人提供涌现认知和交互式语言导向控制方面的巨大潜力。
英文摘要：
We investigate the use of Large Language Models (LLMs) to equip neural robotic agents with human-like social and cognitive competencies, for the purpose of open-ended human-robot conversation and collaboration. We introduce a modular and extensible methodology for grounding an LLM with the sensory perceptions and capabilities of a physical robot, and integrate multiple deep learning models throughout the architecture in a form of system integration. The integrated models encompass various functions such as speech recognition, speech generation, open-vocabulary object detection, human pose estimation, and gesture detection, with the LLM serving as the central text-based coordinating unit. The qualitative and quantitative results demonstrate the huge potential of LLMs in providing emergent cognition and interactive language-oriented control of robots in a natural and social manner.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
