arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2302.09846cs.RO
收藏

基于学习系统动力学的神经最优控制

Neural Optimal Control using Learned System Dynamics

University of Minnesota(明尼苏达大学)

Selim Engin, Volkan Isler

摘要:

研究了动态未知系统的控制律生成问题。我们的方法是用神经网络来表示控制器和价值函数,并用HJB方程中的损失函数来训练它们。在没有已知动态模型的情况下,我们的方法首先从通过在离线过程中与系统交互收集的数据中学习状态转换。然后将学习的过渡函数集成到HJB方程中,并用于在反馈回路中正向模拟由我们的控制器产生的控制信号。与针对单个初始状态优化控制器的轨迹优化方法相比,我们的控制器可以从状态空间的大部分中为初始状态生成接近最优控制信号。与最近的基于模型的强化学习算法相比,我们表明,我们的方法是更有效的样本和训练速度的一个数量级。我们证明了我们的方法在一些任务,包括控制的四旋翼与12个状态变量。

英文摘要:

We study the problem of generating control laws for systems with unknown dynamics. Our approach is to represent the controller and the value function with neural networks, and to train them using loss functions adapted from the Hamilton-Jacobi-Bellman (HJB) equations. In the absence of a known dynamics model, our method first learns the state transitions from data collected by interacting with the system in an offline process. The learned transition function is then integrated to the HJB equations and used to forward simulate the control signals produced by our controller in a feedback loop. In contrast to trajectory optimization methods that optimize the controller for a single initial state, our controller can generate near-optimal control signals for initial states from a large portion of the state space. Compared to recent model-based reinforcement learning algorithms, we show that our method is more sample efficient and trains faster by an order of magnitude. We demonstrate our method in a number of tasks, including the control of a quadrotor with 12 state variables.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑