In this paper, a novel robust tracking control scheme for a general class of discrete-time nonlinear systems affected by unknown bounded uncertainty is presented. By solving a parameterized optimal tracking control problem subject to the unknown nominal system and a suitable cost function, the resulting optimal tracking control policy can ensure closed-loop stability by achieving a sufficiently small tracking error for the original uncertain nonlinear system. The computation of the optimal tracking controller is accomplished through the derivation of a novel Q-function-based $λ$-Policy Iteration algorithm. The proposed algorithm not only enjoys rigorous theoretical guarantees, but also avoids technical weaknesses of conventional reinforcement learning methods. By employing a data-driven, critic-only least squares implementation, the performance of the proposed algorithm is evaluated to the problem of fully-automated, insulin-based, closed-loop glucose control for patients diagnosed with Type 1 and Type 2 Diabetes Mellitus. The U.S. FDA-accepted DMMS.R simulator from the Epsilon Group is used to conduct a comprehensive in silico clinical campaign on a rich set of virtual subjects under completely unannounced meal and exercise settings. Simulation results underline the superior glycaemic behavior achieved by the derived approach, as well as its overall maturity for the design of highly-effective, closed-loop drug delivery systems for personalized medicine.
MPC-guided Imitation Learning of Neural Network Policies for the Artificial Pancreas
Stony Brook University(石溪大学);University of London, Royal Holloway(伦敦大学皇家霍洛威学院)
尽管模型预测控制(MPC)目前是人工胰腺(AP)中胰岛素控制的主要算法,但它通常需要复杂的在线优化,这对于资源受限的医疗设备是不可行的。MPC通常还依赖于状态估计,这是一个容易出错的过程。在本文中,我们介绍了一种新的方法,AP控制,使用模仿学习合成神经网络胰岛素政策MPC计算的演示。这样的策略在计算上是高效的,并且通过在训练时间用完整的状态信息装备MPC,它们可以直接将测量映射到最佳治疗决策中,从而绕过状态估计。我们通过Monte Carlo Dropout应用贝叶斯推理来学习策略,这使我们能够量化预测的不确定性,从而得出更安全的治疗决策。我们表明,我们的控制策略在特定的患者模型下训练容易推广(在模型参数和干扰分布方面)到患者队列,始终优于传统的MPC与状态估计。
Multi-Step Optimal Tracking Control of Unknown Nonzero-Sum Games based on Least Squares and Linear Programming: An Application to a Fully-Automated, Dual-Hormone Artificial Pancreas