Many important complex networks, including critical infrastructure and emerging industrial automation systems, are becoming increasingly intricate webs of interacting feedback control loops. A fundamental concern is to quantify the control properties and performance limitations of the network as a function of its dynamical structure and control architecture. We study performance bounds for networks in terms of optimal feedback control costs. We provide a set of complementary bounds as a function of the system dynamics and actuator structure. For unstable network dynamics, we characterize a tradeoff between feedback control performance and the number of control inputs, in particular showing that optimal cost can increase exponentially with the size of the network. We also derive a bound on the performance of the worst-case actuator subset for stable networks, providing insight into dynamics properties that affect the potential efficacy of actuator selection. We illustrate our results with numerical experiments that analyze performance in regular and random networks.
The distributed optimal output consensus problem for high-order multi-agent systems has been studied recently. In this paper, we further focus on the same problem for high-order multi-agent systems subject to parametric uncertainties and aim at distributed robust controllers by measurement output feedback. We first develop a dynamic compensator to estimate the expected optimal consensus point and convert the problem into several decentralized robust tracking problems. Then, by combining the integral control technique and dirty derivative observer technique, we constructively propose a distribu…
Feedback particle filter (FPF) is a Monte-Carlo (MC) algorithm to approximate the solution of a stochastic filtering problem. In contrast to conventional particle filters, the Bayesian update step in FPF is implemented via a mean-field type feedback control law. The objective for this paper is to situate the development of FPF and related controlled interacting particle system algorithms within the framework of optimal transportation theory. Starting from the simplest setting of the Bayes' update formula, a coupling viewpoint is introduced to construct particle filters. It is shown that the co…
In this work, we revisit the Linear Quadratic Gaussian (LQG) optimal control problem from a behavioral perspective. Motivated by the suitability of behavioral models for data-driven control, we begin with a reformulation of the LQG problem in the space of input-output behaviors and obtain a complete characterization of the optimal solutions. In particular, we show that the optimal LQG controller can be expressed as a static behavioral-feedback gain, thereby eliminating the need for dynamic state estimation characteristic of state space methods. The static form of the optimal LQG gain also make…
Conventional optimal power flow (OPF) solvers assume full observability of the involved system states. However, in practice, there is a lack of reliable system monitoring devices in the distribution networks. To close the gap between the theoretic algorithm design and practical implementation, this work proposes to solve the OPF problems based on the state estimation (SE) feedback for the distribution networks where only a part of the involved system states are physically measured. The SE feedback increases the observability of the under-measured system and provides more accurate system states…
In this paper, a time-optimal feedback solution to the game of two cars, for the case where the pursuer is faster and more agile than the evader, is presented. The concept of continuous subsets of the reachable set is introduced to characterize the time-optimal pursuit-evasion game under feedback strategies. Using these subsets it is shown that, if initially the pursuer is distant enough from the evader, then the feedback saddle point strategies for both the pursuer and the evader are coincident with one of the common tangents from the minimum radius turning circles of the pursuer to the minim…
The communication channels used to convey information between the components of wireless networked control systems (WNCSs) are subject to packet losses due to time-varying fading and interference. We consider a wireless networked control scenario, where the packet loss occurs in both the sensor-controller link (sensing link) and the controller-actuator link (actuation link). Moreover, we consider one time-step delay mode observations of the actuation link. While the problems of state feedback optimal control and stabilizability conditions for systems with one time-step delay mode observations …
Descending predictive feedback (DPF) is an ubiquitous yet unexplained phenomenon in the central nervous system. Motivated by recent observations on motor-related signals in the visual system, we approach this problem from a sensorimotor standpoint and make use of optimal controllers to explain DPF. We define and analyze DPF in the optimal control context, revisiting several control problems (state feedback, full control, and output feedback) to explore conditions that necessitate DPF. We find that even small deviations from the unconstrained state feedback problem (e.g. incomplete sensing, com…
A learning approach for optimal feedback gains for nonlinear continuous time control systems is proposed and analysed. The goal is to establish a rigorous framework for computing approximating optimal feedback gains using neural networks. The approach rests on two main ingredients. First, an optimal control formulation involving an ensemble of trajectories with 'control' variables given by the feedback gain functions. Second, an approximation to the feedback functions via realizations of neural networks. Based on universal approximation properties we prove the existence and convergence of opti…
Indian Institute of Technology Kanpur（印度理工学院坎普尔分校）
We propose a framework for synthesizing a feedback control policy that maximizes the region of attraction (ROA) of a closed-loop nonlinear dynamical system. Our synthesis technique relies on stochastic optimization, which involves computation of an objective function capturing the ROA for a feedback control law. We employ a machine learning technique based on deep neural network to estimate the ROA for a given feedback controller. Overall, our technique is capable of synthesizing a controller co-optimizing traditional control objectives like LQR cost together with ROA. We demonstrate the effic…
In this paper we demonstrate how certain structured feedback gains necessarily emerge as the optimal controller gains in two linear optimal control formulations for multi-agent systems. We consider the cases of linear optimal synchronization and linear optimal centroid stabilization. In the former problem, the considered cost functional integrates squared synchronization error and input, and in the latter, the considered cost functional integrates squared sum of the states and input. Our approach is to view the structures in the feedback gains in terms of a static output feedback with suitable…
University of California, Santa Cruz（加州大学圣克鲁兹分校）；Naval Postgraduate School（海军研究生院）
Recent research has shown that supervised learning can be an effective tool for designing optimal feedback controllers for high-dimensional nonlinear dynamic systems. But the behavior of these neural network (NN) controllers is still not well understood. In this paper we use numerical simulations to demonstrate that typical test accuracy metrics do not effectively capture the ability of an NN controller to stabilize a system. In particular, some NNs with high test accuracy can fail to stabilize the dynamics. To address this we propose two NN architectures which locally approximate a linear qua…
University of California, Santa Cruz（加州大学圣克鲁兹分校）；NASA Langley Research Center（美国国家航空航天局兰利研究中心）；Naval Postgraduate School（海军研究生院）
Recent research shows that supervised learning can be an effective tool for designing near-optimal feedback controllers for high-dimensional nonlinear dynamic systems. But the behavior of neural network controllers is still not well understood. In particular, some neural networks with high test accuracy can fail to even locally stabilize the dynamic system. To address this challenge we propose several novel neural network architectures, which we show guarantee local asymptotic stability while retaining the approximation capacity to learn the optimal feedback policy semi-globally. The proposed …
University of Verona（维罗纳大学）；University of Nottingham（诺丁汉大学）
A supervised learning approach for the solution of large-scale nonlinear stabilization problems is presented. A stabilizing feedback law is trained from a dataset generated from State-dependent Riccati Equation solves. The training phase is enriched by the use gradient information in the loss function, which is weighted through the use of hyperparameters. High-dimensional nonlinear stabilization tests demonstrate that real-time sequential large-scale Algebraic Riccati Equation solves can be substituted by a suitably trained feedforward neural network.
A self-learning approach for optimal feedback gains for finite-horizon nonlinear continuous time control systems is proposed and analysed. It relies on parameter dependent approximations to the optimal value function obtained from a family of universal approximators. The cost functional for the training of an approximate optimal feedback law incorporates two main features. First, it contains the average over the objective functional values of the parametrized feedback control for an ensemble of initial values. Second, it is adapted to exploit the relationship between the maximum principle and …
We consider measurement-feedback control in linear dynamical systems from the perspective of regret minimization. Unlike most prior work in this area, we focus on the problem of designing an online controller which competes with the optimal dynamic sequence of control actions selected in hindsight, instead of the best controller in some specific class of controllers. This formulation of regret is attractive when the environment changes over time and no single controller achieves good performance over the entire time horizon. We show that in the measurement-feedback setting, unlike in the full-…
In this paper, we consider the problem of distributed optimal control of linear dynamical systems with a quadratic cost criterion. We study the case of output feedback control for two interconnected dynamical systems, and show that the linear optimal solution can be obtained from a combination of two uncoupled Riccati equations and two coupled Riccati equations.
We prove the existence of an optimal feedback controller for a stochastic optimization problem constituted by a variation of the Heston model, where a stochastic input process is added in order to minimize a given performance criterion. The stochastic feedback controller is searched by solving a nonlinear backward parabolic equation for which one proves the existence of a martingale solution.
We introduce an alternative approach for the analysis and numerical approximation of the optimal feedback control mapping. It consists in looking at a typical optimal control problem in such a way that feasible controls are mappings depending both in time and space. In this way, the feedback form of the problem is built-in from the very beginning. Optimality conditions are derived for one such optimal mapping, which by construction is the optimal feedback mapping of the problem. In formulating optimality conditions, costates in feedback form are solutions of linear, first-order transport syste…
Optimal Output Feedback Learning Control for Discrete-Time Linear Quadratic Regulation
School of Automation, Beijing Institute of Technology, Beijing 100081, China（自动化学院，北京理工大学，北京）；Department of Chemical Engineering, Queen’s University, Kingston, ON K7L 3N6, Canada（化学工程系，皇后大学，加拿大）
In model predictive control (MPC), an optimal control problem (OCP) is solved for the current state and the first input of the solution, the optimal feedback law, is applied to the system. This procedure requires to solve the OCP in every time step. Recently, a new approach was suggested for linear MPC. The parametric solution of a linear quadratic OCP is a piecewise-affine feedback law. The solution at a point in state space provides an optimal feedback law and a domain on which this law is the optimal solution. As long as the system remains in the domain, the law can be reused and the calcul…
Inspired by online learning, data-dependent regret has recently been proposed as a criterion for controller design. In the regret-optimal control paradigm, causal controllers are designed to minimize regret against a hypothetical optimal noncausal controller, which selects the globally cost-minimizing sequence of control actions given noncausal access to the disturbance sequence. We extend regret-optimal control to the more challenging measurement-feedback setting, where the online controller must compete against the optimal noncausal controller without directly observing the state or the driv…
We consider a type of optimal switching problems with non-uniform execution delays and ramping. Such problems frequently occur in the operation of economical and engineering systems. We first provide a solution to the problem by applying a probabilistic method. The main contribution is, however, a scheme for approximating the optimal control by limiting the information in the state-feedback. In a numerical example the approximation routine gives a considerable computational performance enhancement, when compared to a conventional algorithm.
This paper is concerned with a linear quadratic optimal control for a class of singular Volterra integral equations. Under proper convexity conditions, optimal control uniquely exists, and it could be characterized via Frechet derivative of the quadratic functional in a Hilbert space or via maximum principle type necessary conditions. However, these (equivalent) characterizations have a shortcoming that the current value of the optimal control depends on the future values of the optimal state. Practically, this is not feasible. The main purpose of this paper is to obtain a causal state feedbac…
We give the explicit solution of the optimal control problem which consists in minimizing the epidemic peak in the SIR model when the control is an attenuation factor of the infectious rate, subject to a L 1 budget constraint. The optimal strategy is given as a feedback control which consists in an singular arc maintaining the infected population at a constant level until the immunity threshold is reached, and no intervention outside the singular arc.
A linear-quadratic optimal control problem for a forward stochastic Volterra integral equation (FSVIE, for short) is considered. Under the usual convexity conditions, open-loop optimal control exists, which can be characterized by the optimality system, a coupled system of an FSVIE and a Type-II backward SVIE (BSVIE, for short). To obtain a causal state feedback representation for the open-loop optimal control, a path-dependent Riccati equation for an operator-valued function is introduced, via which the optimality system can be decoupled. In the process of decoupling, a Type-III BSVIE is intr…
In this work, we propose a feedback control based temporal discretization for linear quadratic optimal control problems (LQ problems) governed by controlled mean-field stochastic differential equations. We firstly decompose the original problem into two problems: a stochastic LQ problem and a deterministic one. Secondly, we discretize both LQ problems one after another relying on Riccati equations and control's feedback representations. Then, we prove the convergence rates for the proposed discretization and present an effective algorithm. Finally, a numerical example is provided to support th…
Distributed control problems under some specific information constraints can be formulated as (possibly infinite dimensional) convex optimization problems. The underlying motivation of this work is to develop an understanding of the optimal decision making architecture for such problems. In this paper, we particularly focus on the N-player triangular LQG problems and show that the optimal output feedback controllers have attractive state space realizations. The optimal controller can be synthesized using a set of stabilizing solutions to 2N linearly coupled algebraic Riccati equations, which t…
We revisit the optimal control problem of maximizing biogas production in continuous bio-processes in two directions: 1. over an infinite horizon, 2. with sub-optimal controllers independent of the time horizon. For the first point, we identify a set of optimal controls for the problems with an averaged reward and with a discounted reward when the discount factor goes to 0 and we show that the value functions of both problems are equal. For the finite horizon problem, our approach relies on a framing of the value function by considering a different reward for which the optimal solution has an …
One of the fundamental issues in Control Theory is to design feedback controls. It is well-known that, the purpose of introducing Riccati equations in the deterministic case is to provide the desired feedback controls for linear quadratic control problems. To date, the same problem in the stochastic setting is only partially well-understood. In this paper, we establish the equivalence between the existence of optimal feedback controls for the stochastic linear quadratic control problems with random coefficients and the solvability of the corresponding backward stochastic Riccati equations in a…
The paper is concerned with a general optimization problem for a nonlinear control system, in the presence of a running cost and a terminal cost, with free terminal time. We prove the existence of a patchy feedback whose trajectories are all nearly optimal solutions, with pre-assigned accuracy.
A learning based method for obtaining feedback laws for nonlinear optimal control problems is proposed. The learning problem is posed such that the open loop value function is its optimal solution. This infinite dimensional, function space, problem, is approximated by a polynomial ansatz and its convergence is analyzed. An $\ell_1$ penalty term is employed, which combined with the proximal point method, allows to find sparse solutions for the learning problem. The approach requires multiple evaluations of the elements of the polynomial basis and of their derivatives. In order to do this effici…
We investigate remote estimation over a Gilbert-Elliot channel with feedback. We assume that the channel state is observed by the receiver and fed back to the transmitter with one unit delay. In addition, the transmitter gets ACK/NACK feedback for successful/unsuccessful transmission. Using ideas from team theory, we establish the structure of optimal transmission and estimation strategies and identify a dynamic program to determine optimal strategies with that structure. We then consider first-order autoregressive sources where the noise process has unimodal and symmetric distribution. Using …
A learning technique for finite horizon optimal control problems and its approximation based on polynomials is analyzed. It allows to circumvent, in part, the curse dimensionality which is involved when the feedback law is constructed by using the Hamilton-Jacobi-Bellman (HJB) equation. The convergence of the method is analyzed, while paying special attention to avoid the use of a global Lipschitz condition on the nonlinearity which describes the control system. The practicality and efficiency of the method is illustrated by several examples. For two of them a direct approach based on the HJB …
This paper solves the sparsest feedback selection problem for linear time invariant structured systems, a long-standing open problem in structured systems. We consider structurally cyclic systems with dedicated inputs and outputs. We prove that finding a sparsest feedback selection is of linear complexity for the case of structurally cyclic systems with dedicated inputs and outputs. This problem has received attention recently but key errors in the hardness-proofs have resulted in an erroneous conclusion there. This is also elaborated in this brief paper together with a counter-example.
We present a numerically tractable formulation for computing the optimal control of the class of hybrid dynamical systems whose trajectories are continuous. Our formulation, an extension of existing relaxed-control techniques for switched dynamical systems, incorporates the domain information of each discrete mode as part of the constraints in the optimization problem. Moreover, our numerical results are consistent with phenomena that are particular to hybrid systems, such as the creation of sliding trajectories between discrete modes.
A sparse regression approach for the computation of high-dimensional optimal feedback laws arising in deterministic nonlinear control is proposed. The approach exploits the control-theoretical link between Hamilton-Jacobi-Bellman PDEs characterizing the value function of the optimal control problems, and first-order optimality conditions via Pontryagin's Maximum Principle. The latter is used as a representation formula to recover the value function and its gradient at arbitrary points in the space-time domain through the solution of a two-point boundary value problem. After generating a datase…
In this paper we address the problem to compute state dependent feedback controls for path integral control problems. To this end we generalize the path integral control formula and utilize this to construct parameterized state dependent feedback controllers. In addition, we show a novel relation between control and importance sampling: better control, in terms of control cost, yields more efficient importance sampling, in terms of effective sample size. The optimal control provides a zero-variance estimate.
This paper proposes a feedback guidance law to move the instantaneous impact point (IIP) of a rocket to a desired location. Analytic expressions relating the time derivatives of an IIP with the external acceleration of the rocket are introduced. A near time-optimal feedback-form guidance law to determine the direction of the acceleration for guiding the IIP is developed using the de-rivative expressions. The effectiveness of the proposed guidance law, in comparison with the results of open-loop trajectory optimization, was demonstrated through IIP pointing case studies.
This paper presents necessary and sufficient conditions for deriving a strictly proper dynamic controller which satisfies the negative imaginary output feedback control problem. Our synthesis method divides the output feedback control problem into a state feedback problem and a dual output injection problem. Thus, a controller is formed using the solutions to a pair of dual algebraic Riccati equations. Finally, an illustrative example is offered to show how this method may be applied.
We derive a feedback control law for the control of the downstream flow in a 1-D open channel by manipulating the water flow at an upstream location. We use backstepping for controller design and Lyapunov techniques for stability analysis. Finally, the controller is verified with simulations.
In this work we provide explicit conditions on the existence of optimal feedback controls for stochastic processes with regime-switching. We use the compactification method which needs less regularity conditions on the coefficients of the studied stochastic systems. Two different kinds of controls have been considered: one is the control on the coefficients of the diffusion processes, another is the control on the transition rate matrices of the continuous-time Markov chains. Moreover, the dynamic programming principle is established after showing the continuity of the value function.
