arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2501.09783cs.RO
收藏

GeoManip:将几何约束作为机器人操作通用接口的方法

GeoManip: Geometric Constraints as General Interfaces for Robot Manipulation

University of YYY(YYY大学);Institute of WWW(WWW研究院)

Weiliang Tang, Jia-Hui Pan, Yun-Hui Liu, Masayoshi Tomizuka, Li Erran Li, Chi-Wing Fu, Mingyu Ding

AI总结:

本研究提出无需训练的GeoManip框架,通过大模型生成并解析几何约束、优化轨迹,实现机器人操作的强泛化,具备多类人机交互特性,性能达SOTA且无需高昂训练成本。

摘要:

我们提出了GeoManip框架,该框架可让通用机器人利用从物体及部件关系中推导的核心条件(即几何约束)完成操作任务。例如,切胡萝卜需要遵循一项几何约束:刀刃应与胡萝卜的方向垂直。通过符号语言表示解析这些约束并将其转化为底层动作,GeoManip填补了自然语言与机器人执行之间的鸿沟,在多样甚至未见的任务、物体和场景中具备更强的泛化能力。与需要大量训练的视觉-语言-动作模型不同,GeoManip无需训练,依托大基座模型实现:包含预测分阶段几何约束的约束生成模块,以及识别约束所涉物体部件的几何解析器。随后求解器会优化轨迹,以满足从任务描述和场景中推断出的约束。此外,GeoManip支持上下文学习,并具备五项出色的人机交互特性:实时策略适配、从人类演示中学习、从失败案例中学习、长时序动作规划,以及用于模仿学习的高效数据采集。在仿真和真实场景中的大量评估表明,GeoManip达到了当前最优性能,具备出色的分布外泛化能力,同时避免了成本高昂的模型训练。

英文摘要:

We present GeoManip, a framework to enable generalist robots to leverage essential conditions derived from object and part relationships, as geometric constraints, for robot manipulation. For example, cutting the carrot requires adhering to a geometric constraint: the blade of the knife should be perpendicular to the carrot's direction. By interpreting these constraints through symbolic language representations and translating them into low-level actions, GeoManip bridges the gap between natural language and robotic execution, enabling greater generalizability across diverse even unseen tasks, objects, and scenarios. Unlike vision-language-action models that require extensive training, operates training-free by utilizing large foundational models: a constraint generation module that predicts stage-specific geometric constraints and a geometry parser that identifies object parts involved in these constraints. A solver then optimizes trajectories to satisfy inferred constraints from task descriptions and the scene. Furthermore, GeoManip learns in-context and provides five appealing human-robot interaction features: on-the-fly policy adaptation, learning from human demonstrations, learning from failure cases, long-horizon action planning, and efficient data collection for imitation learning. Extensive evaluations on both simulations and real-world scenarios demonstrate GeoManip's state-of-the-art performance, with superior out-of-distribution generalization while avoiding costly model training.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑