arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2407.02473cs.RO
收藏

面向开放世界物体目标导航的开放场景图

Open Scene Graphs for Open World Object-Goal Navigation

Smart Systems Institute(智能系统研究所);National University of Singapore(新加坡国立大学)

Joel Loo, Zhanxin Wu, David Hsu

AI总结:

提出开放场景图OSG并构建OpenSearch系统,结合基础模型与LLM推理,实现开放世界中自然语言指定物体的零样本物体目标导航。

摘要:

我们如何构建能够执行开放世界语义导航任务的机器人,例如在新颖场景中搜索目标物体?尽管基础模型具备这些任务所需的丰富知识和泛化能力,但仍需要一种合适的场景表征,将它们连接成一个完整的机器人系统。我们通过开放场景图(Open Scene Graphs,OSGs)来解决这一问题;这是一种拓扑—语义表征,能够为这些模型保留并组织开放集场景信息,并且其结构可针对不同环境类型进行配置。我们将基础模型与 OSGs 集成到用于开放世界物体目标导航的 OpenSearch 系统中,该系统能够搜索以自然语言指定的开放集物体,同时可在多种环境和实体形态之间进行零样本泛化。我们的 OSGs 增强了大型语言模型(LLM)的推理能力,使鲁棒的物体目标导航优于现有的 LLM 方法。通过仿真和真实世界实验,我们验证了 OpenSearch 在不同环境、机器人和新颖指令上的泛化能力。

英文摘要:

How can we build robots for open-world semantic navigation tasks, like searching for target objects in novel scenes? While foundation models have the rich knowledge and generalisation needed for these tasks, a suitable scene representation is needed to connect them into a complete robot system. We address this with Open Scene Graphs (OSGs), a topo-semantic representation that retains and organises open-set scene information for these models, and has a structure that can be configured for different environment types. We integrate foundation models and OSGs into the OpenSearch system for Open World Object-Goal Navigation, which is capable of searching for open-set objects specified in natural language, while generalising zero-shot across diverse environments and embodiments. Our OSGs enhance reasoning with Large Language Models (LLM), enabling robust object-goal navigation outperforming existing LLM approaches. Through simulation and real-world experiments, we validate OpenSearch's generalisation across varied environments, robots and novel instructions.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑