arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2311.04060cs.ROcs.LG
收藏

用于鲁棒纯触觉灵巧手内操作的估计器耦合强化学习

Estimator-Coupled Reinforcement Learning for Robust Purely Tactile In-Hand Manipulation

DLR Institute of Robotics and Mechatronics(德国航空航天中心机器人与机电一体化研究所);Technical University of Munich(慕尼黑工业大学);Deggendorf Institute of Technology(德根多夫技术学院)

Lennart Röstel, Johannes Pitz, Leon Sievers, Berthold Bäuml

AI总结:

本文提出一种估计器与控制策略在训练阶段耦合的强化学习方法,解决了纯触觉手内操作中状态估计不准的问题,在仿真和真实环境中均实现了鲁棒的多物体重定向。

摘要:

本文识别并解决了在机器人手内操作中,将基于(强化)学习的控制器与状态估计器进行简单结合所存在的问题。具体而言,我们处理了一项具有挑战性的任务:在手朝下的情况下,实现纯触觉、目标条件下的灵巧手内重定向。由于可用感知信息有限,许多在拥有物体状态完整知识时在仿真中可行的控制策略,无法实现准确的状态估计。因此,分别训练控制器和估计器并在测试时将两者结合会导致性能不佳。我们通过在仿真训练期间就将控制策略与状态估计器进行耦合来解决这一问题。这种方法带来了更鲁棒的状态估计以及该任务上更高的整体性能,同时相比于端到端策略学习保持了可解释性优势。借助我们的GPU加速实现,在单个低成本GPU上从头学习的训练时间中位数仅为6.5小时。在使用DLR-Hand II进行的仿真实验中,针对四种显著不同的物体形状,我们深入分析了我们方法的性能。我们展示了成功的sim2real(仿真到现实)迁移,将四个物体旋转到SO(3)的π/2离散化中的所有24个朝向,这对于如此多样化的形状集合而言是前所未有的。最后,我们的方法可以将一个立方体连续重定向到9个目标(中位数),这在以往方法于该挑战性设置下是无法实现的。

英文摘要:

This paper identifies and addresses the problems with naively combining (reinforcement) learning-based controllers and state estimators for robotic in-hand manipulation. Specifically, we tackle the challenging task of purely tactile, goal-conditioned, dextrous in-hand reorientation with the hand pointing downwards. Due to the limited sensing available, many control strategies that are feasible in simulation when having full knowledge of the object's state do not allow for accurate state estimation. Hence, separately training the controller and the estimator and combining the two at test time leads to poor performance. We solve this problem by coupling the control policy to the state estimator already during training in simulation. This approach leads to more robust state estimation and overall higher performance on the task while maintaining an interpretability advantage over end-to-end policy learning. With our GPU-accelerated implementation, learning from scratch takes a median training time of only 6.5 hours on a single, low-cost GPU. In simulation experiments with the DLR-Hand II and for four significantly different object shapes, we provide an in-depth analysis of the performance of our approach. We demonstrate the successful sim2real transfer by rotating the four objects to all 24 orientations in the $π/2$ discretization of SO(3), which has never been achieved for such a diverse set of shapes. Finally, our method can reorient a cube consecutively to nine goals (median), which was beyond the reach of previous methods in this challenging setting.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑