arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2302.13423cs.ROcs.AIcs.LG
收藏

面向操作任务的仿真与真实强化学习:一种基于共识的方法

Sim-and-Real Reinforcement Learning for Manipulation: A Consensus-based Approach

United Kingdom Atomic Energy Authority(英国原子能管理局);The University of Manchester(曼彻斯特大学)

Wenxing Liu, Hanlin Niu, Wei Pan, Guido Herrmann, Joaquin Carrasco

AI总结:

本文提出基于共识的仿真与真实深度强化学习算法CSAR,用于机械臂抓取与放置任务,以提升仿真与真实联合训练在有限时间和硬件预算下的效率与效果。

摘要:

仿真与真实联合训练是机器人操作中仿真到真实迁移训练的一种有前景的替代方案。然而,当前的仿真与真实联合训练既不够高效,即收敛到最优策略的速度较慢,也不够有效,即需要大量真实世界机器人数据。在有限的时间和硬件预算下,仿真与真实联合训练的性能并不令人满意。本文提出了一种基于共识的仿真与真实深度强化学习算法(CSAR),用于机械臂抓取与放置任务,该算法在仿真和真实世界中均表现出相当的性能。在该算法中,我们在仿真器和真实世界中训练智能体,以获得同时适用于仿真与真实世界的最优策略。我们发现两个有趣的现象:(1)仿真中的最优策略并不是仿真与真实联合训练的最佳策略。(2)仿真智能体越多,仿真与真实联合训练效果越好。实验视频可在以下链接获取:https://youtu.be/mcHJtNIsTEQ。

英文摘要:

Sim-and-real training is a promising alternative to sim-to-real training for robot manipulations. However, the current sim-and-real training is neither efficient, i.e., slow convergence to the optimal policy, nor effective, i.e., sizeable real-world robot data. Given limited time and hardware budgets, the performance of sim-and-real training is not satisfactory. In this paper, we propose a Consensus-based Sim-And-Real deep reinforcement learning algorithm (CSAR) for manipulator pick-and-place tasks, which shows comparable performance in both sim-and-real worlds. In this algorithm, we train the agents in simulators and the real world to get the optimal policies for both sim-and-real worlds. We found two interesting phenomenons: (1) Best policy in simulation is not the best for sim-and-real training. (2) The more simulation agents, the better sim-and-real training. The experimental video is available at: https://youtu.be/mcHJtNIsTEQ.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑