arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2006.09543cs.LGcs.SYeess.SYstat.ML
收藏

数据驱动控制与学习动态:基于模型的方法与无模型方法

Data Driven Control with Learned Dynamics: Model-Based versus Model-Free Approach

Clemson University(克莱姆森大学)

Wenjian Hao, Yiqiang Han

摘要:

本文比较了两种不同类型的数据驱动控制方法,代表基于模型和无模型的方法。一种是最近提出的方法-深度Koopman控制表示(DKRC),它利用深度神经网络将未知的非线性动态系统映射到高维线性系统,这允许采用最先进的控制策略。另一种是经典的无模型控制方法--深度确定性策略梯度(DDPG),它是一种基于Actor-Critic结构的无模型控制方法,已被证明在各种动态系统中是有效的。比较是在OpenAI Gym中进行的,它提供了多个用于基准测试的控制环境。提供了两个示例用于比较,即,经典的倒立摆和月球着陆器连续控制。从实验结果中,我们比较了这两种方法的控制策略和在各种初始化条件下的有效性。我们还检查了从DKRC学习的动态模型与从欧拉-拉格朗日线性化方法导出的分析模型,这证明了从数据驱动的样本有效方法学习的模型对于未知动态的准确性。

英文摘要:

This paper compares two different types of data-driven control methods, representing model-based and model-free approaches. One is a recently proposed method - Deep Koopman Representation for Control (DKRC), which utilizes a deep neural network to map an unknown nonlinear dynamical system to a high-dimensional linear system, which allows for employing state-of-the-art control strategy. The other one is a classic model-free control method based on an actor-critic architecture - Deep Deterministic Policy Gradient (DDPG), which has been proved to be effective in various dynamical systems. The comparison is carried out in OpenAI Gym, which provides multiple control environments for benchmark purposes. Two examples are provided for comparison, i.e., classic Inverted Pendulum and Lunar Lander Continuous Control. From the results of the experiments, we compare these two methods in terms of control strategies and the effectiveness under various initialization conditions. We also examine the learned dynamic model from DKRC with the analytical model derived from the Euler-Lagrange Linearization method, which demonstrates the accuracy in the learned model for unknown dynamics from a data-driven sample-efficient approach.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑