arXiv 2011.084582020-11-17cs.RO

为接触丰富的操作任务学习密集奖励

Learning Dense Rewards for Contact-Rich Manipulation Tasks

X, the Moonshot Factory（X，月球工厂）；University of California, Berkeley（加州大学伯克利分校）；Rice University（莱斯大学）

Zheng Wu, Wenzhao Lian, Vaibhav Unhelkar, Masayoshi Tomizuka, Stefan Schaal

备注：: 8 pages, 5 figures

摘要：

奖励在强化学习中起着至关重要的作用。为了达到期望的策略，设计合适的奖励函数通常需要大量的领域专业知识以及试错。在这里，我们的目标是最大限度地减少设计奖励功能的接触丰富的操作任务的努力。特别是，我们提供了一种方法，能够从机器人的高维观测，如图像和触觉反馈算法提取密集的奖励函数。与最先进的高维奖励学习方法相比，我们的方法不利用对抗性训练，因此不太容易出现相关的训练不稳定性。相反，我们的方法通过以自我监督的方式估计任务进度来学习奖励。我们证明了我们的方法在两个接触丰富的操作任务，即钉孔和USB插入的有效性和效率。实验结果表明，与基线相比，使用学习的奖励函数训练的策略具有更好的性能和更快的收敛速度。

英文摘要：

Rewards play a crucial role in reinforcement learning. To arrive at the desired policy, the design of a suitable reward function often requires significant domain expertise as well as trial-and-error. Here, we aim to minimize the effort involved in designing reward functions for contact-rich manipulation tasks. In particular, we provide an approach capable of extracting dense reward functions algorithmically from robots' high-dimensional observations, such as images and tactile feedback. In contrast to state-of-the-art high-dimensional reward learning methodologies, our approach does not leverage adversarial training, and is thus less prone to the associated training instabilities. Instead, our approach learns rewards by estimating task progress in a self-supervised manner. We demonstrate the effectiveness and efficiency of our approach on two contact-rich manipulation tasks, namely, peg-in-hole and USB insertion. The experimental results indicate that the policies trained with the learned reward function achieves better performance and faster convergence compared to the baselines.

机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。

摘要：

英文摘要：

相关论文