arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2108.04857cs.ROmath.DS
收藏

两种预测强化学习方法的实验研究及其与模型预测控制的比较

An experimental study of two predictive reinforcement learning methods and comparison with model-predictive control

Skolkovo Institute of Science and Technology(斯科尔科沃科学技术研究院)

Dmitrii Dobriborsci, Pavel Osinenko

摘要:

强化学习(RL)已经成功地应用于各种仿真和计算机游戏中。工业相关的应用,如自主移动机器人运动控制,对RL来说有些挑战。本文提出了一种最佳的移动机器人运动控制的预测RL控制器的实验评估。作为比较的基准,使用模型预测控制(MPC)。测试了两种RL方法:推出Q学习,其可以被认为是终端成本为Q函数近似的MPC,以及所谓的堆叠Q学习,其又类似于运行成本代替Q函数近似的MPC。实验的基础是一个移动机器人与差动驱动器(Robotis Turtlebot 3)。实验结果表明,两种RL方法在累积成本方面都优于基线,而堆叠变体表现最好。提供了一系列以前的工作堆叠Q学习,这个特殊的研究支持的想法,MPC与运行成本自适应的启发Q学习具有潜在的性能提升,同时保持良好的性能MPC。

英文摘要:

Reinforcement learning (RL) has been successfully used in various simulations and computer games. Industry-related applications, such as autonomous mobile robot motion control, are somewhat challenging for RL up to date though. This paper presents an experimental evaluation of predictive RL controllers for optimal mobile robot motion control. As a baseline for comparison, model-predictive control (MPC) is used. Two RL methods are tested: a roll-out Q-learning, which may be considered as MPC with terminal cost being a Q-function approximation, and a so-called stacked Q-learning, which in turn is like MPC with the running cost substituted for a Q-function approximation. The experimental foundation is a mobile robot with a differential drive (Robotis Turtlebot3). Experimental results showed that both RL methods beat the baseline in terms of the accumulated cost, whereas the stacked variant performed best. Provided the series of previous works on stacked Q-learning, this particular study supports the idea that MPC with a running cost adaptation inspired by Q-learning possesses potential of performance boost while retaining the nice properties of MPC.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑