在人机交互中使用大语言模型与视觉语言模型达成交互共识
Agreeing to Interact in Human-Robot Interaction using Large Language Models and Vision Language Models
AI总结：
针对人机交互启动决策依赖多情境因素的难题，对比四种基于LLM与VLM的系统设计模式，在含84个场景的测试集上验证了其在明确场景下的有效性，指出开放场景仍存挑战。
摘要：
在人机交互（HRI）领域，交互的启动阶段通常十分复杂。机器人是否应当与人类交流取决于多种情境因素（例如当前人类的活动、交互的紧急程度等）。我们测试了大语言模型（LLM）与视觉语言模型（VLM）能否为该问题提供解决方案。我们对比了四种基于LLM与VLM的不同系统设计模式，并在包含84个人机交互场景的测试集上开展测试。该测试集混合了多个公开数据集，还包含适宜行动具有开放性的场景。我们使用GPT-4o与Phi-3 Vision模型的实验结果表明，当预期行动明确时，LLM与VLM能够处理交互启动问题，但在模型需要平衡人类与机器人双方情境的开放性场景中，仍存在挑战。
英文摘要：
In human-robot interaction (HRI), the beginning of an interaction is often complex. Whether the robot should communicate with the human is dependent on several situational factors (e.g., the current human's activity, urgency of the interaction, etc.). We test whether large language models (LLM) and vision language models (VLM) can provide solutions to this problem. We compare four different system-design patterns using LLMs and VLMs, and test on a test set containing 84 human-robot situations. The test set mixes several publicly available datasets and also includes situations where the appropriate action to take is open-ended. Our results using the GPT-4o and Phi-3 Vision model indicate that LLMs and VLMs are capable of handling interaction beginnings when the desired actions are clear, however, challenge remains in the open-ended situations where the model must balance between the human and robot situation.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
