arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2105.01698eess.SYcs.SY
收藏

基于无模型增量自适应动态规划的近似鲁棒最优调节

Model-Free Incremental Adaptive Dynamic Programming Based Approximate Robust Optimal Regulation

Cong Li, Yongchao Wang, Fangzhou Liu, Qingchen Liu, Martin Buss

摘要:

本文提出了连续时间非线性系统无模型鲁棒最优调节的一种新形式。所提出的基于强化学习的方法,被称为增量自适应动态规划(IADP),利用测量数据,允许近似最优增量控制策略的设计,该控制策略在模型不确定性,环境干扰和输入饱和下增量地稳定受控系统。通过利用时间延迟估计(TDE)技术,我们首先利用传感器数据,以减少一个完整的动态,其中测量数据被采用来构建一个增量的动态,反映了系统的演变,在一个增量的形式的要求。然后,基于自适应动态规划设计近似最优增量控制策略,该策略被实现为一个简化的单临界结构,得到Hamilton-Jacobi-Bellman方程值函数的近似解.此外,对于神经网络,利用经验数据设计了一种保证权值收敛的非策略权值更新律。更重要的是,为了解决无意中引入的TDE误差,我们将TDE误差边界相关项纳入代价函数中,从而在优化过程中衰减TDE误差。给出了系统稳定性证明和权值收敛性证明。数值仿真结果验证了所提IADP的有效性和优越性,特别是在降低控制能量消耗和增强鲁棒性方面。

英文摘要:

This paper presents a new formulation for model-free robust optimal regulation of continuous-time nonlinear systems. The proposed reinforcement learning based approach, referred to as incremental adaptive dynamic programming (IADP), exploits measured data to allow the design of the approximate optimal incremental control strategy, which stabilizes the controlled system incrementally under model uncertainties, environmental disturbances, and input saturation. By leveraging the time delay estimation (TDE) technique, we first exploit sensory data to reduce the requirement of a complete dynamics, where measured data are adopted to construct an incremental dynamics that reflects the system evolution in an incremental form. Then, the resulting incremental dynamics serves to design the approximate optimal incremental control strategy based on adaptive dynamic programming, which is implemented as a simplified single critic structure to get the approximate solution to the value function of the Hamilton-Jacobi-Bellman equation. Furthermore, for the critic artificial neural network, experience data are used to design an off-policy weight update law with guaranteed weight convergence. Rather importantly, to address the unintentionally introduced TDE error, we incorporate a TDE error bound related term into the cost function, whereby the TDE error is attenuated during the optimization process. The system stability proof and the weight convergence proof are provided. Numerical simulations are conducted to validate the effectiveness and superiority of our proposed IADP, especially regarding the reduced control energy expenditure and the enhanced robustness.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑