In this paper, a deep reinforcement learning based method is proposed to obtain optimal policies for optimal infinite-horizon control of probabilistic Boolean control networks (PBCNs). Compared with the existing literatures, the proposed method is model-free, namely, the system model and the initial states needn't to be known. Meanwhile, it is suitable for large-scale PBCNs. First, we establish the connection between deep reinforcement learning and optimal infinite-horizon control, and structure the problem into the framework of the Markov decision process. Then, PBCNs are defined as large-scale or small-scale, depending on whether the memory of the action-values exceeds the RAM of the computer. Based on the newly introduced definition, Q-learning (QL) and double deep Q-network (DDQN) are applied to the optimal infinite-horizon control of small-scale and large-scale PBCNs, respectively. Meanwhile, the optimal state feedback controllers are designed. Finally, two examples are presented, which are a small-scale PBCN with 3 nodes, and a large-scale one with 28 nodes. To verify the convergence of QL and DDQN, the optimal control policy and the optimal action-values, which are obtained from both the algorithms, are compared with the ones based on a model-based method named policy iteration. Meanwhile, the performance of QL is compared with DDQN in the small-scale PBCN.
Reinforcement Learning with Probabilistic Boolean Network Models of Smart Grid Devices
Centro de Ciencias de la Complejidad (C3), Universidad Nacional Autónoma de México(墨西哥国立自治大学复杂性科学中心);Centre de Recerca en Enginyeria Biomèdica, Universitat Politècnica de Catalunya(加泰罗尼亚理工大学生物医学工程研究中心);Instituto de Investigaciones en Matemáticas Aplicadas y en Sistemas, Universidad Nacional Autónoma de México(墨西哥国立自治大学应用数学与系统研究所);Lakeside Labs GmbH(Lakeside Labs 有限公司);Institut de Recerca Sant Joan de Déu(圣琼德乌研究所)
Infinite-Horizon Optimal Control of Switched Boolean Control Networks with Average Cost: An Efficient Graph-Theoretical Approach
本研究探讨具有平均成本准则之切换式布林控制网路之无限时域最佳控制问题。这个问题的一个主要挑战是在处理大规模网络时的高计算成本。我们试图开发一个更有效的和可扩展的方法,从图论的角度来看。首先,一个加权有向图结构称为$\textit{optimal state transition graph}$(OSTG)的建立,其边缘编码的最佳行动,从一个给定的初始状态之间的状态可达的一步之间的转换受到各种约束。然后,我们减少了无限时域最优控制问题的最小平均周期(MMC)问题的OSTG。最后,我们开发了一种新的算法,可以快速找到一个特定的MMC诉诸Karp算法在图论中,然后构造一个最优的开关和控制律的基础上的状态反馈。时间复杂度分析表明,我们的算法可以优于现有的方法在时间效率方面。一个16节点的白血病信号网络被用来作为一个基准来测试其有效性。结果表明,所提出的图论方法的计算效率要高得多:它的运行速度比现有方法快数百甚至数千倍。
On control of discrete-time state-dependent jump linear systems with probabilistic constraints: A receding horizon approach
CRAN–CNRS UMR 7039, Université de Lorraine(洛林大学);Instituto Tecnologico de Hermosillo(埃莫西约理工学院);Instistute of Mathematics “Simon Stoilow”, Romanian Academy(罗马尼亚科学院西蒙·斯托伊洛数学研究所)
Measurement-Induced Boolean Dynamics and Controllability for Quantum Networks
Academy of Mathematics and Systems Science, Chinese Academy of Sciences(中国科学院数学与系统科学研究院);The Australian National University(澳大利亚国立大学);The University of Sydney(悉尼大学)
Receding Horizon Motion Planning for Multi-Agent Systems: A Velocity Obstacle Based Probabilistic Method
NUS Graduate School for Integrative Sciences and Engineering(新加坡国立大学综合科学与工程研究生院);National University of Singapore(新加坡国立大学);University of California, Berkeley(加州大学伯克利分校);Temasek Laboratories(淡马锡实验室)