arXiv 2410.027872025-10-28cs.CVcs.AIcs.CL

基于VLM框架的导航：迈向任意语言目标导航

Navigation with VLM framework: Towards Going to Any Language

Future Network of Intelligence Institute(Shenzhen)（智能未来网络研究所（深圳））；University of Technology Sydney（悉尼技术大学）；Shanghai Jiao Tong University（上海交通大学）；The Chinese University of Hongkong(Shenzhen)（香港中文大学（深圳））

Zecheng Yin, Chonghao Cheng, and Yao Guo, Zhen Li

备注：: under review

阅读 PDF

arXiv 页面 ↗HTML 原文 ↗

AI总结：

针对现有开放场景VLM导航方法计算成本高、依赖物体中心思路或详细带先验指令的问题，提出无需训练的NavVLM框架，以开源VLM为认知核心，仅需简洁目标即可实现智能导航，在仿真多数据集物体任务中SPL达SOTA，真实室内场景验证有效。

摘要：

面向完全开放的语言目标导航并以智能方式探索开放场景，始终是一项重大挑战。近年来，视觉语言模型（Vision Language Models, VLMs）已展现出结合语言与视觉数据进行推理的卓越能力。尽管已有诸多研究聚焦于利用VLM开展开放场景导航，但这类方法往往计算成本高昂，依赖以物体为中心的思路，或是需要包含环境先验的详细人类指令。\n我们提出了Navigation with VLM（NavVLM，基于VLM的导航），这是一个无需训练的框架，借助开源VLM让机器人能够有效完成导航，即便目标是符合人类表达习惯的语言指令，比如开放场景中的抽象地点、动作或是特定物体。NavVLM以VLM作为认知核心来感知环境信息，并持续提供探索引导，仅需简洁的目标描述即可实现智能导航，无需附带环境先验的详细指令。\n我们通过仿真实验与真实世界实验对NavVLM进行了评估验证。在仿真环境中，我们的框架在Matterport 3D（MP3D）、Habitat Matterport 3D（HM3D）以及Gibson的高细节环境下的特定物体任务中，取得了路径长度加权成功率（Success weighted by Path Length, SPL）的当前最优性能。从导航回合的结果来看，NavVLM展现了面向任意开放集语言目标的导航能力。在真实世界验证中，我们在室内场景的实体机器人上验证了该框架的有效性。

英文摘要：

Navigating towards fully open language goals and exploring open scenes in an intelligent way have always raised significant challenges. Recently, Vision Language Models (VLMs) have demonstrated remarkable capabilities to reason with both language and visual data. Although many works have focused on leveraging VLMs for navigation in open scenes, they often require high computational cost, rely on object-centric approaches, or depend on environmental priors in detailed human instructions. We introduce Navigation with VLM (NavVLM), a training-free framework that harnesses open-source VLMs to enable robots to navigate effectively, even for human-friendly language goal such as abstract places, actions, or specific objects in open scenes. NavVLM leverages the VLM as its cognitive core to perceive environmental information and constantly provides exploration guidance achieving intelligent navigation with only a neat target rather than a detailed instruction with environment prior. We evaluated and validated NavVLM in both simulation and real-world experiments. In simulation, our framework achieves state-of-the-art performance in Success weighted by Path Length (SPL) on object-specifc tasks in richly detailed environments from Matterport 3D (MP3D), Habitat Matterport 3D (HM3D) and Gibson. With navigation episode reported, NavVLM demonstrates the capabilities to navigate towards any open-set languages. In real-world validation, we validated our framework's effectiveness in real-world robot at indoor scene.

机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。