arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2410.12802cs.RO
收藏

通过视觉-语言模型解决机器人导航对话中的位置歧义

Resolving Positional Ambiguity in Dialogues by Vision-Language Models for Robot Navigation

College of Artificial Intelligence, National Yang Ming Chiao Tung University (NYCU)(国立阳明交通大学人工智能学院)

Kuan-Lin Chen, Tzu-Ti Wei, Li-Tzu Yeh, Elaine Kao, Yu-Chee Tseng, Jen-Jee Chen

AI总结:

本文提出利用大型语言模型和视觉-语言模型进行多轮对话,并通过两级映射(自然语言到唯一物体ID,再到三维深度图)解决室内机器人导航指令中的位置歧义问题,首次将基础模型应用于该问题。

摘要:

我们考虑一种自主导航机器人,它能够通过自然语言接受人类指令,以在室内环境中提供服务。这些自然语言指令可能包含时间、位置、物体和动作等组成部分。然而,我们观察到,此类指令中的位置组成部分通常指向环境中可能具有不同程度位置歧义的物体。例如,当房间中存在多把同类型的椅子时,指令“去椅子那里!”可能具有歧义。为了消除这些指令的歧义,我们采用了一个大型语言模型和一个大型视觉-语言模型与用户进行多轮对话。我们提出了一种两级方法,利用视觉-语言模型将自然语言中的含义映射到图像中的唯一物体ID,然后再将唯一物体ID映射到三维深度图,从而使机器人能够从其当前位置导航到目标位置。据我们所知,这是首个将基础模型与位置歧义问题联系起来的工作。

英文摘要:

We consider an autonomous navigation robot that can accept human commands through natural language to provide services in an indoor environment. These natural language commands may include time, position, object, and action components. However, we observe that the positional components within such commands usually refer to objects in the environment that may contain different levels of positional ambiguity. For example, the command "Go to the chair!" may be ambiguous when there are multiple chairs of the same type in a room. In order to disambiguate these commands, we employ a large language model and a large vision-language model to conduct multiple turns of conversations with the user. We propose a two-level approach that utilizes a vision-language model to map the meanings in natural language to a unique object ID in images and then performs another mapping from the unique object ID to a 3D depth map, thereby allowing the robot to navigate from its current position to the target position. To the best of our knowledge, this is the first work linking foundation models to the positional ambiguity issue.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑