The linear-quadratic controller is one of the fundamental problems in control theory. The optimal solution is a linear controller that requires access to the state of the entire system at any given time. When considering a network system, this renders the optimal controller a centralized one. The interconnected nature of a network system often demands a distributed controller, where different components of the system are controlled based only on local information. Unlike the classical centralized case, obtaining the optimal distributed controller is usually an intractable problem. Thus, we adopt a graph neural network (GNN) as a parametrization of distributed controllers. GNNs are naturally local and have distributed architectures, making them well suited for learning nonlinear distributed controllers. By casting the linear-quadratic problem as a self-supervised learning problem, we are able to find the best GNN-based distributed controller. We also derive sufficient conditions for the resulting closed-loop system to be stable. We run extensive simulations to study the performance of GNN-based distributed controllers and showcase that they are a computationally efficient parametrization with scalability and transferability capabilities.
Controlling network systems has become a problem of paramount importance. In this paper, we consider a distributed linear-quadratic problem and propose the use of graph neural networks (GNNs) to parametrize and design a distributed controller for network systems. GNNs exhibit many desirable properties, such as being naturally distributed and scalable. We cast the distributed linear-quadratic problem as a self-supervised learning problem, which is then used to train the GNN-based controllers. We also obtain sufficient conditions for the resulting closed-loop system to be input-state stable, and…
An optimal control law for networked control systems with a discrete-time linear time-invariant (LTI) system as plant and networks between sensor and controller as well as between controller and actuator is proposed. This controller is designed by solving an optimization problem that is a generalization of the optimization problem used to obtain the Linear Quadratic Regulator (LQR) for deterministic discrete-time LTI systems. The networks are represented by random delays and drop outs of transmitted data packets.
Classical linear quadratic (LQ) control centers around linear time-invariant (LTI) systems, where the control-state pairs introduce a quadratic cost with time-invariant parameters. Recent advancement in online optimization and control has provided novel tools to study LQ problems that are robust to time-varying cost parameters. Inspired by this line of research, we study the distributed online LQ problem for identical LTI systems. Consider a multi-agent network where each agent is modeled as an LTI system. The LTI systems are associated with decoupled, time-varying quadratic costs that are rev…
In this paper we consider a recently developed distributed optimization algorithm based on gradient tracking. We propose a system theory framework to analyze its structural properties on a preliminary, quadratic optimization set-up. Specifically, we focus on a scenario in which agents in a static network want to cooperatively minimize the sum of quadratic cost functions. We show that the gradient tracking distributed algorithm for the investigated program can be viewed as a sparse closed-loop linear system in which the dynamic state-feedback controller includes consensus matrices and optimizat…
John A. Paulson School of Engineering and Applied Sciences, Harvard University（哈佛大学约翰·A·保尔森工程与应用科学学院）
This paper considers a distributed reinforcement learning problem for decentralized linear quadratic control with partial state observations and local costs. We propose a Zero-Order Distributed Policy Optimization algorithm (ZODPO) that learns linear local controllers in a distributed fashion, leveraging the ideas of policy gradient, zero-order optimization and consensus algorithms. In ZODPO, each agent estimates the global cost by consensus, and then conducts local policy gradient in parallel based on zero-order gradient estimation. ZODPO only requires limited communication and storage even i…
Mirror descent (MD) is a powerful first-order optimization technique that subsumes several optimization algorithms including gradient descent (GD). In this work, we develop a semi-definite programming (SDP) framework to analyze the convergence rate of MD in centralized and distributed settings under both strongly convex and non-strongly convex assumptions. We view MD with a dynamical system lens and leverage quadratic constraints (QCs) to provide explicit convergence rates based on Lyapunov stability. For centralized MD under strongly convex assumption, we develop a SDP that certifies exponent…
We study the distributed Linear Quadratic Gaussian (LQG) control problem in discrete-time and finite-horizon, where the controller depends linearly on the history of the outputs and it is required to lie in a given subspace, e.g. to possess a certain sparsity pattern. It is well-known that this problem can be solved with convex programming within the Youla domain if and only if a condition known as Quadratic Invariance (QI) holds. In this paper, we first show that given QI sparsity constraints, one can directly descend the gradient of the cost function within the domain of output-feedback cont…
Near-Optimal Distributed Linear-Quadratic Regulator for Networked Systems
Mathematics and Computer Science Division, Argonne National Laboratory（阿贡国家实验室数学与计算机科学部）；California Institute of Technology（加州理工学院）；Department of Electrical and Computer Engineering, Carnegie Mellon University（卡内基梅隆大学电气与计算机工程系）；Department of Statistics, University of Chicago（芝加哥大学统计系）
In this paper, a cooperative Linear Quadratic Regulator (LQR) problem is investigated for multi-input systems, where each input is generated by an agent in a network. The input matrices are different and locally possessed by the corresponding agents respectively, which can be regarded as different ways for agents to control the multi-input system. By embedding a fully distributed information fusion strategy, a novel cooperative LQR-based controller is proposed. Each agent only needs to communicate with its neighbors, rather than sharing information globally in a network. Moreover, only the joi…
A method is presented to analyze the stability of feedback systems with neural network controllers. Two stability theorems are given to prove asymptotic stability and to compute an ellipsoidal inner-approximation to the region of attraction (ROA). The first theorem addresses linear time-invariant systems, and merges Lyapunov theory with local (sector) quadratic constraints to bound the nonlinear activation functions in the neural network. The second theorem allows the system to include perturbations such as unmodeled dynamics, slope-restricted nonlinearities, and time delay, using integral qua…
Distributed Adaptive Neural Interval Observers for Unknown Nonlinear Systems
Faculty of Mechanical Engineering, Ho Chi Minh City University of Technology (HCMUT), Vietnam National University Ho Chi Minh City (VNU-HCM)（胡志明市理工大学机械工程学院，越南国立大学胡志明市分校）；Department of Mathematics and Statistics, University of New Mexico（新墨西哥大学数学与统计系）
This paper presents a distributed optimization algorithm tailored to solve optimization problems arising in smart grids. In detail, we propose a variant of the Augmented Lagrangian based Alternating Direction Inexact Newton (ALADIN) method, which comes along with global convergence guarantees for the considered class of linear-quadratic optimization problems. We establish local quadratic convergence of the proposed scheme and elaborate its advantages compared to the Alternating Direction Method of Multipliers (ADMM). In particular, we show that, at the cost of more communication, ALADIN requir…
We study model-free learning methods for the output-feedback Linear Quadratic (LQ) control problem in finite-horizon subject to subspace constraints on the control policy. Subspace constraints naturally arise in the field of distributed control and present a significant challenge in the sense that standard model-based optimization and learning leads to intractable numerical programs in general. Building upon recent results in zeroth-order optimization, we establish model-free sample-complexity bounds for the class of distributed LQ problems where a local gradient dominance constant exists on a…
In this work, we propose a distributed adaptive observer for a class of nonlinear networked systems inspired by biophysical neural network models. Neural systems learn by adjusting intrinsic and synaptic weights in a distributed fashion, with neuronal membrane voltages carrying information from neighbouring neurons in the network. We show that this learning principle can be used to design an adaptive observer based on a decentralized learning rule that greatly reduces the number of observer states required for exponential convergence of parameter estimates. This novel design is relevant for bi…
Distributed state estimation is examined for a sensor network tasked with reconstructing a system's state through the use of a distributed and event-triggered observer. Each agent in the sensor network employs a deep neural network (DNN) to approximate the uncertain nonlinear dynamics of the system, which is trained using a multiple timescale approach. Specifically, the outer weights of each DNN are updated online using a Lyapunov-based gradient descent update law, while the inner weights and biases are trained offline using a supervised learning method and collected input-output data. The obs…
Distributed data collection is a fundamental task in open systems. In such networks, data is aggregated across a network to produce a single aggregated result at a source device. Though self-stabilizing, algorithms performing data collection can produce large overestimates in the transient phase. For example, in [1] we demonstrated that in a line graph, a switch of sources after initial stabilization may produce overestimates that are quadratic in the network diameter. We also proposed monotonic filtering as a strategy for removing such large overestimates. Monotonic filtering prevents the tra…
Quadratic Characterizations for Reachability Analysis of Neural Networks
Kevin T. Crofton Department of Aerospace and Ocean Engineering, Virginia Tech（凯文·T·克罗夫顿航空航天与海洋工程系，弗吉尼亚理工学院）；Federation ENAC ISAE-SUPAERO ONERA, Universite de Toulouse（ENAC ISAE-SUPAERO ONERA联盟，图卢兹大学）
When designing large-scale distributed controllers, the information-sharing constraints between sub-controllers, as defined by a communication topology interconnecting them, are as important as the controller itself. Controllers implemented using dense topologies typically outperform those implemented using sparse topologies, but it is also desirable to minimize the cost of controller deployment. Motivated by the above, we introduce a compact but expressive graph recurrent neural network (GRNN) parameterization of distributed controllers that is well suited for distributed controller and commu…
This paper presents a novel distributed active set method for model predictive control of linear systems. The method combines a primal active set strategy with a decentralized conjugate gradient method to solve convex quadratic programs. An advantage of the proposed method compared to existing distributed model predictive algorithms is the primal feasibility of the iterates. Numerical results show that the proposed method can compete with the alternating direction method of multipliers in terms of communication requirements for a chain of masses example.
This paper addresses distributed parameter estimation in randomized one-hidden-layer neural networks. A group of agents sequentially receive measurements of an unknown parameter that is only partially observable to them. In this paper, we present a fully distributed estimation algorithm where agents exchange local estimates with their neighbors to collectively identify the true value of the parameter. We prove that this distributed update provides an asymptotically unbiased estimator of the unknown parameter, i.e., the first moment of the expected global error converges to zero asymptotically.…
University of Illinois, Urbana, IL 61801 USA（伊利诺伊大学厄巴纳-香槟分校）；Error Corp（Error公司）；Sandia National Laboratories（沙ia国家实验室）；Department of Electrical and Computer Engineering and the Coordinated Science Laboratory, University of Illinois, Urbana, IL 61801, USA（伊利诺伊大学厄巴纳-香槟分校电子与计算机工程系和协调科学实验室）
Southwest Minzu University（西南民族大学）；Sichuan Vocational College of Finance and Economics（四川财经职业学院）；Henan Polytechnic University（河南理工大学）
In this paper, a family of novel diffusion adaptive estimation algorithm is proposed from the asymmetric cost function perspective by combining diffusion strategy and the linear-linear cost (LLC), quadratic-quadratic cost (QQC), and linear-exponential cost (LEC), at all distributed network nodes, and named diffusion LLCLMS (DLLCLMS), diffusion QQCLMS (DQQCLMS), and diffusion LECLMS (DLECLMS), respectively. Then the stability of mean estimation error and computational complexity of those three diffusion algorithms are analyzed theoretically. Finally, several experiment simulation results are de…
This paper investigates the distributed Kalman filtering (DKF) from distributed optimization viewpoint. Motivated by the fact that Kalman filtering is a maximum a posteriori estimation (MAP) problem, which is a quadratic optimization problem, we reformulate DKF problem as a consensus optimization problem, resulting in that it can be solved by many existing distributed optimization algorithms. A new DKF algorithm employing the dual ascent method is proposed, and its stability is proved under mild assumptions. The performance of the proposed algorithm is evaluated through numerical experiments.
Graph Neural Network-Based Distributed Optimal Control for Linear Networked Systems: An Online Distributed Training Approach
Department of Electrical Engineering, University of Notre Dame（电气工程系，诺丁汉大学）；Department of Electrical and Computer Engineering, Stevens Institute of Technology（电气与计算机工程系，史蒂文斯理工学院）
École Polytechnique Fédérale de Lausanne (EPFL)（洛桑联邦理工学院）
Large-scale cyber-physical systems require that control policies are distributed, that is, that they only rely on local real-time measurements and communication with neighboring agents. Optimal Distributed Control (ODC) problems are, however, highly intractable even in seemingly simple cases. Recent work has thus proposed training Neural Network (NN) distributed controllers. A main challenge of NN controllers is that they are not dependable during and after training, that is, the closed-loop system may be unstable, and the training may fail due to vanishing and exploding gradients. In this pap…
