arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 1811.04006cs.LGcs.SYeess.SYstat.ML
收藏

基于可达性的安全学习用于最优控制问题

Reachability-based safe learning for optimal control problem

Department of Computer Science, Systems and Communication, University of Milano Bicocca(米兰Bicocca大学计算机科学与通信系)

Stanislav Fedorov, Antonio Candelieri

AI总结:

本文提出了一种结合系统部分已知状态空间模型和未知动态作为加性有界扰动的安全学习方法,旨在通过安全集选择最优动作以实现目标集,同时在学习过程中更新扰动并提升最优控制的鲁棒性。

摘要:

在本文中,我们寻求一种整合安全性的学习方法,该方法依赖于部分已知的系统状态空间模型,并将未知动态视为加性有界扰动。我们引入了一个框架,用于在存在扰动的情况下安全地学习控制策略。基于已知模型部分,算法可以在满足安全保持条件的情况下,选择最优动作以追求目标集。在一些学习回合后,扰动可以根据现实数据进行更新。为此,对收集的扰动样本进行高斯过程回归。由于现实世界的不稳定性,例如摩擦或导电性随温度的变化,我们期望获得更鲁棒的最优控制问题解决方案。为了评估上述方法,我们选择倒立摆作为基准模型。所提出的算法能够学习到不违反预设安全约束的策略。当将其与探索设置结合时,观察到性能有所提升,从而确保在安全集内学习到最优策略。最后,我们概述了一些超出本文范围的未来研究方向。

英文摘要:

In this work we seek for an approach to integrate safety in the learning process that relies on a partly known state-space model of the system and regards the unknown dynamics as an additive bounded disturbance. We introduce a framework for safely learning a control strategy for a given system with an additive disturbance. On the basis of the known part of the model, a safe set in which the system can learn safely, the algorithm can choose optimal actions for pursuing the target set as long as the safety-preserving condition is satisfied. After some learning episodes, the disturbance can be updated based on real-world data. To this end, Gaussian Process regression is conducted on the collected disturbance samples. Since the unstable nature of the law of the real world, for example, change of friction or conductivity with the temperature, we expect to have the more robust solution of optimal control problem. For evaluation of approach described above we choose an inverted pendulum as a benchmark model. The proposed algorithm manages to learn a policy that does not violate the pre-specified safety constraints. Observed performance is improved when it was incorporated exploration set up to make sure that an optimal policy is learned everywhere in the safe set. Finally, we outline some promising directions for future research beyond the scope of this paper.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑