arXiv 2510.269152026-09-22cs.ROcs.AI

非结构化环境中基于具身生成智能的异构机器人协作

Heterogeneous Robot Collaboration in Unstructured Environments with Grounded Generative Intelligence

Dept. of Electrical and Computer Engineering at Texas A&M University（德克萨斯A&M大学电气与计算机工程系）；GRASP Laboratory at the University of Pennsylvania（宾夕法尼亚大学GRASP实验室）

Zachary Ravichandran, Fernando Cladera, Ankit Prabhu, Jason Hughes, Carlos Nieto-Granda, Varun Murali, Camillo Taylor, George J. Pappas, Vijay Kumar

备注：: Accepted to the International Conference on Intelligent Robots and Systems (IROS) 2026

阅读 PDF

arXiv 页面 ↗HTML 原文 ↗

AI总结：

本文提出SPINE-HT框架，通过三阶段过程将LLM推理能力扎根于异构机器人团队上下文中，实现非结构化环境下的自然语言任务分配与在线反馈细化，仿真与真实实验均显著提升了任务成功率。

摘要：

在现实环境中运行的异构机器人团队通常必须完成需要协作并适应在线获取信息的复杂任务。由于机器人团队经常在非结构化环境——没有先验地图的、不确定的开放世界环境中运行，因此子任务必须基于机器人能力和物理世界。虽然异构团队通常是为固定规格而设计的，但生成智能为实现能够完成自然语言描述的广泛任务的团队提供了可能。然而，当前基于大型语言模型（LLM）的团队协作方法通常假设环境是结构良好且已知的，这限制了它们在非结构化环境中的部署。我们提出了SPINE-HT框架，该框架通过三阶段过程将LLM的推理能力扎根于异构机器人团队的上下文中，从而解决了这些限制。给定描述任务目标和团队能力的语言规格，LLM会生成扎根的子任务并验证其可行性。然后根据可通行性或感知等能力将子任务分配给机器人，并根据在线运行期间收集的反馈进行细化。在具有闭环感知和控制的仿真实验中，与先前的基于LLM的异构团队协作方法相比，我们的框架实现了近两倍的成功率。在使用Clearpath Jackal、Clearpath Husky、Boston Dynamics Spot和高空UAV进行的真实世界实验中，我们的方法在需要推理机器人能力并利用在线反馈细化子任务的任务中达到了87%的成功率。更多信息请访问https://zacravichandran.github.io/SPINE-HT。

英文摘要：

While heterogeneous teams have typically been designed for well-specified missions with known semantics, generative intelligence, i.e., large language models (LLMs) and vision language models (VLMs), opens the possibility of teams that infer mission-relevant semantics and subtasks given high-level natural language specifications and environmental context. However, current LLM- and VLM-enabled teaming methods typically assume well-structured and known environments, limiting performance in complex real-world settings. We address these limitations via SPINE-HT, a framework that grounds the reasoning abilities of LLMs in the evolving context of a heterogeneous robot team through a three-stage process. Given mission specifications and team capabilities in natural language, an LLM infers necessary subtasks. These subtasks are validated for feasibility and then assigned to robots based on capabilities such as traversability or perception. Finally, the framework uses feedback (semantic maps and task outcomes) acquired by the robots to update subtasks online. In simulation experiments with closed-loop perception and control, our framework achieves nearly twice the success rate compared to prior LLM-enabled heterogeneous teaming approaches. In real-world experiments with a Clearpath Jackal, a Clearpath Husky, a Boston Dynamics Spot, and a high-altitude UAV, our method achieves an 87.5% success rate in missions requiring reasoning about robot capabilities and refining subtasks with online feedback. More information is provided at: https://zacravichandran.github.io/SPINE-HT.

机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。