arXiv 2409.164842024-10-02cs.RO

BehAV：基于行为规则引导与视觉语言模型（VLMs）的室外场景机器人导航自主方法

BehAV: Behavioral Rule Guided Autonomy Using VLMs for Robot Navigation in Outdoor Scenes

University of Maryland（马里兰大学）；A. James Clark School of Engineering, University of Maryland（马里兰大学A.詹姆斯·克拉克工程学院）

Kasun Weerakoon, Mohamed Elnoor, Gershom Seneviratne, Vignesh Rajagopal, Senthil Hariharan Arul, Jing Liang, Mohamed Khalid M Jaffar, Dinesh Manocha

阅读 PDF

arXiv 页面 ↗HTML 原文 ↗

AI总结：

BehAV利用大语言模型解析人类指令，结合视觉语言模型构建行为成本图，并通过无约束MPC规划器在室外场景中实现兼顾地标到达与行为规则遵循的机器人导航，成功率较现有方法提升40%。

摘要：

我们提出BehAV，一种在室外场景中由人类指令引导并利用视觉语言模型（VLMs）实现自主机器人导航的新方法。我们的方法使用大语言模型（LLM）解释人类命令，并将指令分类为导航指南和行为指南。导航指南由方向性命令（例如“向前移动直到”）和相关地标（例如“带有蓝色窗户的建筑”）组成，而行为指南包含规制性动作（例如“保持在”）及其对应对象（例如“人行道”）。我们利用VLMs的零样本场景理解能力，从RGB图像中估计地标位置以用于机器人导航。此外，我们引入一种新颖的场景表示，利用VLMs将行为规则锚定到行为成本图中。该成本图编码场景中行为对象的存在，并根据其规制性动作分配成本。行为成本图与基于LiDAR的占据地图集成以用于导航。为了在遵循所指示行为的同时在室外场景中导航，我们提出一种基于无约束模型预测控制（MPC）的规划器，该规划器同时优先到达地标和遵循行为指南。我们在四足机器人上跨多种真实世界场景评估了BehAV的性能，结果显示，以Frechet距离衡量，与人类遥操作动作的一致性提高了22.49%，并且与最先进方法相比，导航成功率提高了40%。

英文摘要：

We present BehAV, a novel approach for autonomous robot navigation in outdoor scenes guided by human instructions and leveraging Vision Language Models (VLMs). Our method interprets human commands using a Large Language Model (LLM) and categorizes the instructions into navigation and behavioral guidelines. Navigation guidelines consist of directional commands (e.g., "move forward until") and associated landmarks (e.g., "the building with blue windows"), while behavioral guidelines encompass regulatory actions (e.g., "stay on") and their corresponding objects (e.g., "pavements"). We use VLMs for their zero-shot scene understanding capabilities to estimate landmark locations from RGB images for robot navigation. Further, we introduce a novel scene representation that utilizes VLMs to ground behavioral rules into a behavioral cost map. This cost map encodes the presence of behavioral objects within the scene and assigns costs based on their regulatory actions. The behavioral cost map is integrated with a LiDAR-based occupancy map for navigation. To navigate outdoor scenes while adhering to the instructed behaviors, we present an unconstrained Model Predictive Control (MPC)-based planner that prioritizes both reaching landmarks and following behavioral guidelines. We evaluate the performance of BehAV on a quadruped robot across diverse real-world scenarios, demonstrating a 22.49% improvement in alignment with human-teleoperated actions, as measured by Frechet distance, and achieving a 40% higher navigation success rate compared to state-of-the-art methods.

机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。