arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2009.00476eess.SYcs.SY
收藏

基于非策略风险敏感强化学习的最优跟踪控制

Off Policy Risk Sensitive Reinforcement Learning Based Optimal Tracking Control with Prescribe Performances

C. Li, Y. Wang, F. Liu, M. Buss

摘要:

针对最优跟踪控制问题,提出了一种基于非策略强化学习的控制策略,使系统在学习过程中达到规定的全状态性能。将最优跟踪控制问题转化为基于辅助系统的最优调节问题。在一个优化框架下,将规定性能的要求转化为约束满足问题,并通过风险敏感的状态惩罚项来处理。为了得到Hamilton Jacobi Bellman方程的近似解,提出了一种结合当前数据和经验数据的非策略自适应批评学习结构。通过使用经验数据,建议的批评学习代理的权重估计更新律保证权重收敛到实际值。与一般的权值收敛需要引入外部信号以满足激励持续性条件的方法相比,该方法具有一定的实用性。给出了闭环系统稳定性和权值收敛性的证明。仿真结果验证了所提出的基于非策略风险敏感强化学习控制策略的有效性。

英文摘要:

An off policy reinforcement learning based control strategy is developed for the optimal tracking control problem to achieve the prescribed performance of full states during the learning process. The optimal tracking control problem is converted as an optimal regulation problem based on an auxiliary system. The requirements of prescribed performances are transformed into constraint satisfaction problems that are dealt with by risk sensitive state penalty terms under an optimization framework. To get approximated solutions of the Hamilton Jacobi Bellman equation, an off policy adaptive critic learning architecture is developed by using current data and experience data together. By using experience data, the proposed weight estimation update law of the critic learning agent guarantees weight convergence to the actual value. This technique enjoys practicability comparing with common methods that need to incorporate external signals to satisfy the persistence of excitation condition for weight convergence. The proofs of stability and weight convergence of the closed loop system are provided. Simulation results reveal the validity of the proposed off policy risk sensitive reinforcement learning based control strategy.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑