基于视觉语言模型的腿式机器人适应中的常识推理
Commonsense Reasoning for Legged Robot Adaptation with Vision-Language Models
AI总结：
本文提出VLM-PC系统，利用视觉语言模型的常识推理，通过上下文适应与多技能前瞻规划，使腿式机器人在死胡同、攀爬等复杂真实场景中实现自主自适应行为选择。
摘要：
腿式机器人在物理上能够导航各种不同的环境并克服广泛的障碍。例如，在搜索与救援任务中，腿式机器人可以爬过废墟、爬行穿过缝隙，并从死胡同中导航出来。然而，机器人的控制器需要智能地应对这些多样的障碍，这要求成功处理意外和不寻常的场景。这对当前的学习方法提出了一个开放性挑战，这些方法在没有大量人类监督的情况下，往往难以泛化到意外情况的长尾分布。为了解决这一问题，我们研究了如何利用视觉语言模型（VLMs）关于世界结构的广泛知识和常识推理能力，来帮助腿式机器人处理困难、模糊的情境。我们提出了一个系统，VLM-Predictive Control（VLM-PC），结合了两个我们发现对于利用VLMs激发即时、自适应行为选择至关重要的关键组件：（1）基于先前机器人交互的上下文适应，以及（2）规划未来多个技能并重新规划。我们在Go1四足机器人上，对几个具有挑战性的现实世界障碍赛道（包括死胡同、攀爬和爬行）评估了VLM-PC。我们的实验表明，通过对交互历史和未来计划进行推理，VLMs使机器人能够在广泛的复杂场景中自主感知、导航和行动，而这些场景原本需要针对特定环境的工程或人类指导。
英文摘要：
Legged robots are physically capable of navigating a diverse variety of environments and overcoming a wide range of obstructions. For example, in a search and rescue mission, a legged robot could climb over debris, crawl through gaps, and navigate out of dead ends. However, the robot's controller needs to respond intelligently to such varied obstacles, and this requires handling unexpected and unusual scenarios successfully. This presents an open challenge to current learning methods, which often struggle with generalization to the long tail of unexpected situations without heavy human supervision. To address this issue, we investigate how to leverage the broad knowledge about the structure of the world and commonsense reasoning capabilities of vision-language models (VLMs) to aid legged robots in handling difficult, ambiguous situations. We propose a system, VLM-Predictive Control (VLM-PC), combining two key components that we find to be crucial for eliciting on-the-fly, adaptive behavior selection with VLMs: (1) in-context adaptation over previous robot interactions and (2) planning multiple skills into the future and replanning. We evaluate VLM-PC on several challenging real-world obstacle courses, involving dead ends and climbing and crawling, on a Go1 quadruped robot. Our experiments show that by reasoning over the history of interactions and future plans, VLMs enable the robot to autonomously perceive, navigate, and act in a wide range of complex scenarios that would otherwise require environment-specific engineering or human guidance.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
