arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2304.02052cs.RO
收藏

学习抓取序列策略的在线增强以实现更具适应性和数据高效的灵巧操作

Online augmentation of learned grasp sequence policies for more adaptable and data-efficient in-hand manipulation

University of Washington(华盛顿大学);Honda Research Institute(本田研究院)

Ethan K. Gordon, Rana Soltani Zarrin

AI总结:

针对灵巧工具操作中抓取序列的适应性与数据效率问题,提出利用离散动态规划原理增强强化学习策略,通过环境近似执行树搜索与策略展开,在极少在线计算下提升性能并加速训练。

摘要:

在使用工具时,用于拾取、重新调整姿势以及将其保持在所需任务合适姿态的抓取方式可能截然不同。因此,自主灵巧工具操作的一个关键挑战是寻找一系列抓取动作,以促进工具使用过程的每一步,同时持续保持力闭合和稳定性。由于接触动力学建模的复杂性,强化学习(RL)技术可以在受高度参数化物理模型约束的连续空间中提供解决方案。然而,这些技术在适应性和数据效率方面存在权衡。在测试阶段,工具属性、期望轨迹和期望施加力可能与训练场景存在显著差异。要适应这些情况,需要更多数据或计算成本高昂的在线策略更新。在本研究中,我们应用离散动态规划(DP)原理,利用领域知识来增强RL性能。具体而言,我们首先设计了一个计算简单的环境近似模型。然后,我们在物理仿真中证明,利用该近似模型执行树搜索(即前瞻)和策略展开,能够以极少的额外在线计算量改进RL生成的抓取序列策略。此外,我们还表明,使用DP导出的离散化问题解决方案来预训练深度RL网络,可以加速策略训练。

英文摘要:

When using a tool, the grasps used for picking it up, reposing, and holding it in a suitable pose for the desired task could be distinct. Therefore, a key challenge for autonomous in-hand tool manipulation is finding a sequence of grasps that facilitates every step of the tool use process while continuously maintaining force closure and stability. Due to the complexity of modeling the contact dynamics, reinforcement learning (RL) techniques can provide a solution in this continuous space subject to highly parameterized physical models. However, these techniques impose a trade-off in adaptability and data efficiency. At test time the tool properties, desired trajectory, and desired application forces could differ substantially from training scenarios. Adapting to this necessitates more data or computationally expensive online policy updates. In this work, we apply the principles of discrete dynamic programming (DP) to augment RL performance with domain knowledge. Specifically, we first design a computationally simple approximation of our environment. We then demonstrate in physical simulation that performing tree searches (i.e., lookaheads) and policy rollouts with this approximation can improve an RL-derived grasp sequence policy with minimal additional online computation. Additionally, we show that pretraining a deep RL network with the DP-derived solution to the discretized problem can speed up policy training.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑