arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2202.11280cs.RO
收藏

基于场景视觉观察和先前动作Q值预测的机器人多步操作策略学习

Learning Multi-step Robotic Manipulation Policies from Visual Observation of Scene and Q-value Predictions of Previous Action

Rochester Institute of Technology（罗切斯特理工学院）；OSARO Inc（OSARO公司）；Siemens Corporation, Corporate Technology（西门子公司企业技术部）

Sulabh Kumra, Shirin Joshi, Ferat Sahin

备注：
7 pages, 3 figures, IEEE Conference on Robotics and Automation (ICRA) 2022. arXiv admin note: substantial text overlap with arXiv:2103.01434
阅读 PDF
arXiv 页面 ↗HTML 原文 ↗

摘要：

在这项工作中，我们专注于涉及长期规划的多步操作任务，并考虑进度逆转。这些任务交织在一起，高级推理包括可以达到的预期状态，以实现整体任务和低级推理，决定什么行动将产生这些状态。我们提出了一个高效的先前动作条件机器人操作网络（PAC-RoManNet）样本，以学习动作-价值函数并根据场景的视觉观察和先前动作的动作-价值预测来预测操作动作候选者。我们定义了一个基于任务进度的高斯（TPG）奖励函数，该函数基于导致成功的运动基元和朝向整体任务目标的进度的动作来计算奖励。为了平衡勘探/开采的比例，我们引入了一个损失调整勘探（LAE）的政策，根据玻尔兹曼分布的损失估计，从行动候选人确定行动。我们通过训练PAC-RoManNet来学习模拟和现实世界中的几个具有挑战性的多步机器人操作任务，从而证明了我们方法的有效性。实验结果表明，我们的方法优于现有的方法，并达到了最先进的性能，在成功率和行动效率。消融研究表明，TPG和LAE对于多块堆叠等任务特别有益。在Ravens-10基准任务上的其他实验表明，所提出的PAC-RoManNet具有良好的通用性。

英文摘要：

In this work, we focus on multi-step manipulation tasks that involve long-horizon planning and considers progress reversal. Such tasks interlace high-level reasoning that consists of the expected states that can be attained to achieve an overall task and low-level reasoning that decides what actions will yield these states. We propose a sample efficient Previous Action Conditioned Robotic Manipulation Network (PAC-RoManNet) to learn the action-value functions and predict manipulation action candidates from visual observation of the scene and action-value predictions of the previous action. We define a Task Progress based Gaussian (TPG) reward function that computes the reward based on actions that lead to successful motion primitives and progress towards the overall task goal. To balance the ratio of exploration/exploitation, we introduce a Loss Adjusted Exploration (LAE) policy that determines actions from the action candidates according to the Boltzmann distribution of loss estimates. We demonstrate the effectiveness of our approach by training PAC-RoManNet to learn several challenging multi-step robotic manipulation tasks in both simulation and real-world. Experimental results show that our method outperforms the existing methods and achieves state-of-the-art performance in terms of success rate and action efficiency. The ablation studies show that TPG and LAE are especially beneficial for tasks like multiple block stacking. Additional experiments on Ravens-10 benchmark tasks suggest good generalizability of the proposed PAC-RoManNet.

机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。

↑