arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2603.09971cs.RO
收藏

TiPToP:一种用于机器人操作的模块化开放式词汇规划系统

TiPToP: A Modular Open-Vocabulary Robot Manipulation System That Plans

William Shen, Nishanth Kumar, Sahit Chintalapudi, Ryan Lindeborg, Jie Wang, Christopher Watson, Edward Hu, Jing Cao, Dinesh Jayaraman, Leslie Pack Kaelbling, Tomás Lozano-Pérez

AI总结:

TiPToP是一种模块化开放式词汇规划系统,结合预训练视觉模型与任务规划器,通过自然语言指令和RGB图像直接解决多步骤操作任务,实现实时高效的操作规划。

摘要:

我们提出了TiPToP,一种可扩展的模块化系统,它结合了预训练的视觉基础模型与现有的任务和运动规划器(TAMP),直接从输入的RGB图像和自然语言指令解决多步骤操作任务。我们的系统旨在简单易用:可以在标准DROID设置下在不到一小时内安装和运行,并且可以以最小的努力适应新的型号。我们评估了TiPToP——它不需要任何机器人数据——在28个桌面操作任务的模拟和现实世界中,并发现其表现与或优于π_{0.5}-DROID,一个经过350小时特定型号演示微调的视觉-语言-动作(VLA)模型。TiPToP的模块化架构使我们能够分析系统在组件层面的失败模式。我们分析了173次试验的结果,并确定了改进的方向。我们发布TiPToP开源,以进一步研究模块化操作系统以及学习与规划之间的更紧密整合。项目网站和代码:https://tiptop-robot.github.io

英文摘要:

We present TiPToP, a modular manipulation system that integrates pretrained foundation models with a GPU-accelerated Task and Motion Planner to solve tasks directly from RGB images and natural language. TiPToP composes perception, planning, and execution modules and requires no robot training data. It can be deployed on a standard DROID setup in under an hour and adapted to new embodiments with minimal effort. We evaluate TiPToP against $π_{0.5}\text{-DROID}$, a state-of-the-art VLA fine-tuned on 350 hours of demonstrations, across two real-world DROID setups (one operated by an external team) and simulation, where TiPToP attains a higher average success rate and faster average completion time. We also evaluate on the MolmoSpaces benchmark, where TiPToP ranks first overall on pick and pick-and-place tasks among methods not trained on in-distribution data. We further show that TiPToP's modularity enables us to trace failures to specific components, revealing where to target improvements. We release TiPToP open-source to serve as a reproducible baseline and to enable further research on modular manipulation systems. Project website and code: https://tiptop-robot.github.io

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑