arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2004.14501eess.SYcs.LGcs.SY
收藏

基于奇异摄动近似的降维强化学习控制

Reduced-Dimensional Reinforcement Learning Control using Singular Perturbation Approximations

North Carolina State University(北卡罗来纳州立大学);Oklahoma State University(俄克拉荷马州立大学)

Sayak Mukherjee, He Bai, Aranya Chakrabortty

摘要:

针对线性定常奇异摄动系统,提出了一种基于无模型降维强化学习的最优控制设计方法。首先,我们提出了一个基于状态反馈和输出反馈的RL控制设计的一般SP系统的状态和输入矩阵未知。我们利用潜在的时间尺度分离属性的植物学习一个线性二次型调节器(LQR),只有其缓慢的动态,从而节省了大量的学习时间相比,传统的全维RL控制器。我们使用SP逼近定理分析了设计的次优性,并提供了闭环稳定的充分条件。此后,我们将这两种设计扩展到聚类多智能体共识网络,其中SP属性通过聚类反映。我们开发了集中式和集群式块分散RL控制器,这样的网络,在减少的维度。我们演示了这些控制器的实现使用相关的数值例子的模拟的细节,并将其与传统的RL设计,以显示我们的方法的计算效益。

英文摘要:

We present a set of model-free, reduced-dimensional reinforcement learning (RL) based optimal control designs for linear time-invariant singularly perturbed (SP) systems. We first present a state-feedback and output-feedback based RL control design for a generic SP system with unknown state and input matrices. We take advantage of the underlying time-scale separation property of the plant to learn a linear quadratic regulator (LQR) for only its slow dynamics, thereby saving a significant amount of learning time compared to the conventional full-dimensional RL controller. We analyze the sub-optimality of the design using SP approximation theorems and provide sufficient conditions for closed-loop stability. Thereafter, we extend both designs to clustered multi-agent consensus networks, where the SP property reflects through clustering. We develop both centralized and cluster-wise block-decentralized RL controllers for such networks, in reduced dimensions. We demonstrate the details of the implementation of these controllers using simulations of relevant numerical examples and compare them with conventional RL designs to show the computational benefits of our approach.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑