arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 1901.07905eess.SYcs.SY
收藏

强化学习船舶自动驾驶:基于样本高效与模型预测控制的方法

Reinforcement Learning Ship Autopilot: Sample efficient and Model Predictive Control-based Approach

Yunduan Cui, Shigeki Osaki, Takamitsu Matsubara

AI总结:

针对真实尺寸船舶自主控制中环境不确定性高和真实采样成本极高的问题,提出结合样本高效基于模型的强化学习与MPC的SPMPC方法,通过迭代学习高斯过程动力学模型在闭环控制中高效更新控制信号,在仿真和真实船舶自动驾驶任务中均取得成功。

摘要:

本研究聚焦于为一项具有挑战性的任务开发强化学习系统:真实尺寸船舶的自主控制。该任务的困难源于充满挑战的海洋环境中巨大的不确定性,以及使用真实船舶进行探索和采样的极高成本。为此,我们探索了一种基于高斯过程(GP)的新型强化学习方法,该方法结合了样本高效的基于模型的强化学习与模型预测控制(MPC)。我们的方法名为样本高效概率模型预测控制(SPMPC),该方法迭代地学习高斯过程动力学模型,并利用该模型在MPC闭环控制中高效更新控制信号。我们构建了一个使用SPMPC的系统,以高效学习自动驾驶任务。在基于真实船舶驾驶数据建模的仿真中研究其性能后,所提出的系统在无需人类示范的情况下,成功学会了在自动驾驶任务中驾驶一艘配备单引擎以及测量GPS、速度、方向和风速传感器的真实尺寸船舶。

英文摘要:

In this research we focus on developing a reinforcement learning system for a challenging task: autonomous control of a real-sized boat, with difficulties arising from large uncertainties in the challenging ocean environment and the extremely high cost of exploring and sampling with a real boat. To this end, we explore a novel Gaussian processes (GP) based reinforcement learning approach that combines sample-efficient model-based reinforcement learning and model predictive control (MPC). Our approach, sample-efficient probabilistic model predictive control (SPMPC), iteratively learns a Gaussian process dynamics model and uses it to efficiently update control signals within the MPC closed control loop. A system using SPMPC is built to efficiently learn an autopilot task. After investigating its performance in a simulation modeled upon real boat driving data, the proposed system successfully learns to drive a real-sized boat equipped with a single engine and sensors measuring GPS, speed, direction, and wind in an autopilot task without human demonstration.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑