arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2410.07403cs.RO
收藏

混合方法求解长时序任务导向灵巧操作的可行性研究

On the Feasibility of A Mixed-Method Approach for Solving Long Horizon Task-Oriented Dexterous Manipulation

Virginia Tech(弗吉尼亚理工大学);Honda Research Institute(本田研究所)

Shaunak A. Mehta, Rana Soltani Zarrin

AI总结:

针对灵巧手长时序工具操作问题,提出结合模仿学习、强化学习与基于模型控制的混合方法及新RL师生框架,在仿真与真实世界中均优于单一强化学习方法。

摘要:

在现实世界中使用灵巧手进行工具的掌内操作是文献中一个研究不足的问题。与立方体或圆柱体等常用物体相比,工具具有更复杂的几何形状和更大的尺寸;此外,任务导向的掌内工具操作涉及许多需要顺序执行的子任务。这可能包括接近工具、抓取工具、在手中重新定向工具(有或没有重新抓取)以达到适合工具使用的期望最终抓取姿态,以及将工具移动到期望姿态。使用灵巧手进行长时序操作的研究相当有限,现有工作侧重于使用强化学习(RL)等方法学习各个子任务,并组合不同子任务的策略以执行长时序任务。然而,通常单一方法可能并不适用于所有子任务,当使用多指手操作具有复杂几何形状的物体(如工具)时,这一问题更加突出。在本文中,我们研究了使用混合方法来解决工具使用的长时序任务,并使用了模仿学习、强化学习和基于模型的控制。我们还讨论了一种新的基于RL的师生框架,该框架将真实世界数据结合到离线训练中。我们表明,我们针对每个子任务提出的方法在仿真中跨不同子任务及执行长时序任务时均优于通常采用的强化学习方法。最后,我们展示了向现实世界的成功可转移性。

英文摘要:

In-hand manipulation of tools using dexterous hands in real-world is an underexplored problem in the literature. In addition to more complex geometry and larger size of the tools compared to more commonly used objects like cubes or cylinders, task oriented in-hand tool manipulation involves many sub-tasks to be performed sequentially. This may involve reaching to the tool, picking it up, reorienting it in hand with or without regrasping to reach to a desired final grasp appropriate for the tool usage, and carrying the tool to the desired pose. Research on long-horizon manipulation using dexterous hands is rather limited and the existing work focus on learning the individual sub-tasks using a method like reinforcement learning (RL) and combine the policies for different subtasks to perform a long horizon task. However, in general a single method may not be the best for all the sub-tasks, and this can be more pronounced when dealing with multi-fingered hands manipulating objects with complex geometry like tools. In this paper, we investigate the use of a mixed-method approach to solve for the long-horizon task of tool usage and we use imitation learning, reinforcement learning and model based control. We also discuss a new RL-based teacher-student framework that combines real world data into offline training. We show that our proposed approach for each subtask outperforms the commonly adopted reinforcement learning approach across different subtasks and in performing the long horizon task in simulation. Finally we show the successful transferability to real world.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑