arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2510.25754cs.RO

GET-USE:通过模拟具身扩展学习用于双臂移动操作的通用工具使用方法

GET-USE: Learning Generalized Tool Usage for Bimanual Mobile Manipulation via Simulated Embodiment Extensions

Stanford University(斯坦福大学);University of Texas at Austin(德克萨斯大学奥斯汀分校)

Bohan Wu, Paul de La Sayette, Li Fei-Fei, Roberto Martín-Martín

AI总结:

针对现有机器人工具使用方法泛化性不足、无法从多物体中选最优工具的问题,提出GeT-USE两步框架,先在模拟中探索具身扩展习得工具几何知识,再迁移至真实机器人,在22自由度双臂移动操作任务中成功率领先30-60%。

摘要:

以可泛化的方式将随机物体用作工具的能力,是当今机器人智能中提升其多功能性与问题解决能力的缺失环节。当前最先进的机器人工具使用方法,专注于为特定任务程序化生成或众包工具数据集,以学习如何抓取和操作这些工具完成该任务。但这些方法假定仅提供一个物体,且通过正确抓取即可完成任务;当存在多个可用物体时,它们无法识别、抓取并使用最适合任务的物体,尤其是在最优工具缺失的情况下。\n本研究提出GeT-USE,这是一种两步流程,通过先在模拟中学习扩展机器人的具身结构,再将习得的策略迁移到真实机器人的视觉运动策略中,实现真实机器人的通用工具使用。我们的核心洞见是:通过在模拟中探索机器人的具身扩展(即构建新型末端执行器),机器人能够识别对任务最有利的通用工具几何形状。随后可将习得的几何知识进行蒸馏,通过选择并使用现实世界中可用的最佳物体作为工具,完成通用工具使用任务。\n在一台具有22个自由度(DOFs)的真实机器人上,GeT-USE在三项基于视觉的双臂移动操作工具使用任务中,成功率较当前最优方法高出30-60%。

英文摘要:

The ability to use random objects as tools in a generalizable manner is a missing piece in robots' intelligence today to boost their versatility and problem-solving capabilities. State-of-the-art robotic tool usage methods focused on procedurally generating or crowd-sourcing datasets of tools for a task to learn how to grasp and manipulate them for that task. However, these methods assume that only one object is provided and that it is possible, with the correct grasp, to perform the task; they are not capable of identifying, grasping, and using the best object for a task when many are available, especially when the optimal tool is absent. In this work, we propose GeT-USE, a two-step procedure that learns to perform real-robot generalized tool usage by learning first to extend the robot's embodiment in simulation and then transferring the learned strategies to real-robot visuomotor policies. Our key insight is that by exploring a robot's embodiment extensions (i.e., building new end-effectors) in simulation, the robot can identify the general tool geometries most beneficial for a task. This learned geometric knowledge can then be distilled to perform generalized tool usage tasks by selecting and using the best available real-world object as tool. On a real robot with 22 degrees of freedom (DOFs), GeT-USE outperforms state-of-the-art methods by 30-60% success rates across three vision-based bimanual mobile manipulation tool-usage tasks.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑