arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2109.03100cs.RO
收藏

基于策略梯度方法的乒乓球机器人最优击球学习

Optimal Stroke Learning with Policy Gradient Approach for Robotic Table Tennis

Eberhard Karls University Tübingen(图宾根大学)

Yapeng Gao, Jonas Tebbe, Andreas Zell

摘要:

学习打乒乓球对机器人来说是一项具有挑战性的任务,因为需要各种各样的击球方式。最近的进展表明,深度强化学习(RL)能够在模拟环境中成功学习最佳动作。然而,RL在真实场景中的适用性仍然有限,由于高勘探工作。在这项工作中,我们提出了一个现实的仿真环境中,多个模型的球的动力学和机器人的运动学。代替训练端到端RL模型,提出了一种新的具有TD3骨干的策略梯度方法,以基于击球时球的预测状态来学习球拍击球。在实验中,我们表明,所提出的方法显着优于现有的RL方法在模拟。此外,为了跨越从模拟到现实的领域,我们采用了一种有效的再训练方法,并在三个真实场景中进行了测试。结果成功率为98%,距离误差约为24.9 cm。总培训时间约为1.5小时。

英文摘要:

Learning to play table tennis is a challenging task for robots, as a wide variety of strokes required. Recent advances have shown that deep Reinforcement Learning (RL) is able to successfully learn the optimal actions in a simulated environment. However, the applicability of RL in real scenarios remains limited due to the high exploration effort. In this work, we propose a realistic simulation environment in which multiple models are built for the dynamics of the ball and the kinematics of the robot. Instead of training an end-to-end RL model, a novel policy gradient approach with TD3 backbone is proposed to learn the racket strokes based on the predicted state of the ball at the hitting time. In the experiments, we show that the proposed approach significantly outperforms the existing RL methods in simulation. Furthermore, to cross the domain from simulation to reality, we adopt an efficient retraining method and test it in three real scenarios. The resulting success rate is 98% and the distance error is around 24.9 cm. The total training time is about 1.5 hours.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑