具有随机分组乱序的网络化控制系统的可证强化学习
Provable Reinforcement Learning for Networked Control Systems with Stochastic Packet Disordering
AI总结：
针对随机分组乱序的线性网络化控制系统，本文建立带唯一镇定解的最优控制框架，推导MARE，并提出可证收敛的离线算法及在线无模型强化学习算法。
摘要：
本文针对具有随机分组乱序（PD）的线性网络化控制系统，构建了一个随机最优控制问题，并证明其存在唯一的镇定解。该问题通过提出强化学习算法得以解决。首先给出一种测量方法来处理PD并计算最新的控制输入。将具有随机PD的网络化控制系统（NCSs）建模为随机NCSs。随后，在给定代价函数的情况下，在该表述框架内推导了一个修正代数Riccati方程（MARE）。我们提出离线策略迭代和值迭代算法来求解MARE，并给出了可证明的收敛性。这两种算法需要知道NCS动力学和PD概率。为解除这一要求，我们进一步设计了在线无模型离策略（off-policy）算法和Q-learning算法，并采用PD概率的在线估计方法。两种无模型算法均利用实时系统状态、控制输入和PD概率估计来求解最优控制问题。最后，仿真结果验证了所提出的表述和算法。
英文摘要：
This paper formulates a stochastic optimal control problem for linear networked control systems featuring stochastic packet disordering with a unique stabilizing solution certified. The problem is solved by proposing reinforcement learning algorithms. A measurement method is first presented to deal with PD and calculate the newest control input. The NCSs with stochastic PD are modeled as stochastic NCSs. Then, given a cost function, a modified algebraic Riccati equation is derived within the formulation. We propose offline policy iteration and value iteration algorithms to solve the MARE associated with provable convergence. These two algorithms require knowledge of NCS dynamics and PD probabilities. To release that, we further design online model-free off-policy and Q-learning algorithms with an online estimation method for PD probability. Both model-free algorithms solve the optimal control problem using real-time system states, control inputs, and PD probability estimates. Simulation results verify the proposed formulation and algorithms at last.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
