arXiv 2103.054052021-03cs.RO

杂波环境下面向目标的推抓协同的有效学习

Efficient learning of goal-oriented push-grasping synergy in clutter

Zhejiang University（浙江大学）

Kechun Xu, Hongxiang Yu, Qianen Lai, Yue Wang, Rong Xiong

摘要：

我们专注于目标导向的抓取任务，其中一个机器人应该抓住一个预先分配的目标对象的混乱，需要一些预抓动作，如推动，使稳定的把握。然而，在这个任务中，机器人只有在成功抓住目标物体时才能从环境中获得积极的奖励。此外，关节推和抓延长了动作序列，加剧了奖励延迟的问题。因此，样本效率低下仍然是这项任务的主要挑战。本文提出了一种具有高样本效率的目标条件分层强化学习方法，用于学习在杂乱环境中抓取特定目标的推抓取策略。在我们的工作中，样本效率提高了两种手段。首先，我们使用一个目标条件的机制，目标重新标记，以丰富重放缓冲区。其次，将推策略和抓取策略分别作为生成器和学习器，在抓取学习器的监督下训练推策略，从而增加推策略的奖励。针对两种策略训练设置不同导致的分布不匹配问题，增加了一个交替训练阶段，依次学习推和抓。仿真和真实环境下的实验表明，该方法能够快速学习有效的推取策略，在任务完成率和目标抓取成功率方面均优于现有方法，且运动次数较少.此外，我们验证了我们的系统也可以适应目标不可知的条件下，更好的性能。请注意，我们的系统可以在不进行任何微调的情况下转移到现实世界。我们的代码可在https://github.com/xukechun/Efficient_goal-oriented_push-grasping_synergy上获得。

英文摘要：

We focus on the task of goal-oriented grasping, in which a robot is supposed to grasp a pre-assigned goal object in clutter and needs some pre-grasp actions such as pushes to enable stable grasps. However, in this task, the robot gets positive rewards from environment only when successfully grasping the goal object. Besides, joint pushing and grasping elongates the action sequence, compounding the problem of reward delay. Thus, sample inefficiency remains a main challenge in this task. In this paper, a goal-conditioned hierarchical reinforcement learning formulation with high sample efficiency is proposed to learn a push-grasping policy for grasping a specific object in clutter. In our work, sample efficiency is improved by two means. First, we use a goal-conditioned mechanism by goal relabeling to enrich the replay buffer. Second, the pushing and grasping policies are respectively regarded as a generator and a discriminator and the pushing policy is trained with supervision of the grasping discriminator, thus densifying pushing rewards. To deal with the problem of distribution mismatch caused by different training settings of two policies, an alternating training stage is added to learn pushing and grasping in turn. A series of experiments carried out in simulation and real world indicate that our method can quickly learn effective pushing and grasping policies and outperforms existing methods in task completion rate and goal grasp success rate by less times of motion. Furthermore, we validate that our system can also adapt to goal-agnostic conditions with better performance. Note that our system can be transferred to the real world without any fine-tuning. Our code is available at https://github.com/xukechun/Efficient_goal-oriented_push-grasping_synergy.

机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。

摘要：

英文摘要：

相关论文