This paper studies an infinite horizon optimal control problem for discrete-time linear systems and quadratic criteria, both with random parameters which are independent and identically distributed with respect to time. A classical approach is to solve an algebraic Riccati equation that involves mathematical expectations and requires certain statistical information of the parameters. In this paper, we propose an online iterative algorithm in the spirit of Q-learning for the situation where only one random sample of parameters emerges at each time step. The first theorem proves the equivalence of three properties: the convergence of the learning sequence, the well-posedness of the control problem, and the solvability of the algebraic Riccati equation. The second theorem shows that the adaptive feedback control in terms of the learning sequence stabilizes the system as long as the control problem is well-posed. Numerical examples are presented to illustrate our results.
The paper addresses the stabilization of nonlinear systems with semi-quadratic cost: quadratic with respect to controls and nonlinear for state variables. Paper presents the effective new feedback synthesis procedure. The novel feedback design procedure is based on the ideas borrowed from nonlinear optics and the theory of semi-classical asymptotics.
In this paper, we present Lyapunov-based adaptive controllers for the practical (or real) stabilization of a perturbed chain of integrators with bounded uncertainties. We refer to such controllers as Adaptive Higher Order Sliding Mode (AHOSM) controllers since they are designed for nonlinear SISO systems with bounded uncertainties such that the uncertainty bounds are unknown. Our main result states that, given any neighborhood N of the origin, we determine a controller insuring, for every uncertainty bounds, that every trajectory of the corresponding closed loop system enters N and eventually …
This paper is concerned with uniform stabilization and social optimality for general mean field linear quadratic control systems, where subsystems are coupled via individual dynamics and costs, and the state weight is not assumed with the definiteness condition. For the finite-horizon problem, we first obtain a set of forward-backward stochastic differential equations (FBSDEs) from variational analysis, and construct a feedback-type control by decoupling the FBSDEs. For the infinite-horizon problem, by using solutions to two Riccati equations, we design a set of decentralized control laws, whi…
In the present work, a simple algorithm for stabilizing an unknown linear time-invariant system is proposed, assuming only that this system is stabilizable. The suggested algorithm is based on first performing a partial identification of the system and then stabilizing the controllable subsystem. It should be emphasized that our approach does not depend on any prior model knowledge and requires only a minimal number of samples.
In this paper the finite-time stabilization problem is solved for a linear time-varying system with unknown control direction by exploiting a modified version of the classical extremum seeking algorithm. We propose to use a suitable oscillatory input to modify the system dynamics, at least in an average sense, so as to satisfy a Differential Linear Matrix Inequality (DLMI) condition which in turns guarantees that the system's state remains inside a prescribed time varying hyper-ellipsoid in the state space. The finite-time stability (FTS) of the averaged dynamics implies the FTS of the origina…
This work studies data-driven switched controller design for discrete-time switched linear systems. Instead of having access to the full system dynamics, an initialization phase is performed, during which noiseless measurements of the state and the input are collected for each mode. Under certain conditions on these measurements, we develop a stabilizing switched controller for the switched system. To be precise, the controller switches between identifying the active mode of the system and applying a predetermined stabilizing feedback. We prove that if the system switches according to certain …
We investigate the problem of stabilizing an unknown networked linear system under communication constraints and adversarial disturbances. We propose the first provably stabilizing algorithm for the problem. The algorithm uses a distributed version of nested convex body chasing to maintain a consistent estimate of the network dynamics and applies system level synthesis to determine a distributed controller based on this estimated model. Our approach avoids the need for system identification and accommodates a broad class of communication delay while being fully distributed and scaling favorabl…
This paper mainly investigates the optimal control and stabilization problems for linear discrete-time Markov jump systems. The general case for the finite-horizon optimal controller is considered, where the input weighting matrix in the performance index is just required to be positive semi-definite. The necessary and sufficient condition for the existence of the optimal controller in finite-horizon is given explicitly from a set of coupled difference Riccati equations (CDRE). One of the key techniques is to solve the forward and backward stochastic difference equation (FDSDE) which is obtain…
The prescribed-time stabilization problem for a general class of nonlinear systems with unknown input gain and appended dynamics (with unmeasured state) is addressed. Unlike the asymptotic stabilization problem, the prescribed-time stabilization objective requires convergence of the state to the origin in a finite time that can be arbitrarily picked (i.e., prescribed) by the control system designer irrespective of the initial condition of the system. The class of systems considered is allowed to have general nonlinear uncertain terms throughout the system dynamics as well as uncertain appended…
In this paper, we study the problem of stabilizing switched linear systems when only limited information about the state and the mode of the system is available, which occurs in many applications involving networked switched systems (such as cyber-physical systems, IoT, etc.). First, we show that switched linear systems with arbitrary switching, i.e., with no constraint on the switching signal, are in general not stabilizable with a finite data rate. Then, drawing on this result, we restrict our attention to systems satisfying a fairly mild slow-switching assumption, in the sense that the swit…
We consider the problem of voltage stability and reactive power balancing in islanded small-scale electrical networks outfitted with DC/AC inverters ("microgrids"). A droop-like voltage feedback controller is proposed which is quadratic in the local voltage magnitude, allowing for the application of circuit-theoretic analysis techniques to the closed-loop system. The operating points of the closed-loop microgrid are in exact correspondence with the solutions of a reduced power flow equation, and we provide explicit solutions and small-signal stability analyses under several static and dynamic …
This paper studies switching stabilization problems for general switched nonlinear systems. A piecewise smooth control-Lyapunov function (PSCLF) approach is proposed and a constructive way to design a stabilizing switching law is developed. The switching law is constructed via the directional derivatives of the PSCLF with a careful discussion on various technical issues that may occur on the nonsmooth surfaces. Sufficient conditions are derived to ensure stability of the closed-loop Filippov solutions including possible sliding motions. The proposed PSCLF approach contains many existing result…
This paper is concerned with the existence and uniqueness of weak solutions to the Cauchy-Dirichlet problem of backward stochastic partial differential equations (BSPDEs) with nonhomogeneous terms of quadratic growth in both the gradient of the first unknown and the second unknown. As an example, we consider a non-Markovian stochastic optimal control problem with cost functional formulated by a quadratic BSDE, where the corresponding value function satisfies the above quadratic BSPDE.
A design framework recently has been developed to stabilize interconnected multiagent systems in a distributed manner, and systematically capture the architectural aspect of cyber-physical systems. Such a control theoretic framework, however, results in a stabilization protocol which is passive with respect to the cyber attacks and conservative regarding the guaranteed level of resiliency. We treat the control layer topology and stabilization gains as the degrees of freedom, and develop a mixed control and cybersecurity design framework to address the above concerns. From a control perspective…
In this paper, we present Lyapunov-based robust and adaptive controllers for the finite time stabilization of a perturbed chain of integrators with bounded uncertainties. The proposed controllers can be designed for integrator chains of any arbitrary length. The uncertainty bounds are known in the robust control problem whereas they are unknown in the adaptive control problem. Both controllers are developed from a class of finite time stabilization controllers for pure integrator chains. Lyapunov-based design permits to calculate upper bound on convergence time.
In a recent paper we have shown how to learn controllers for unknown linear systems using finite-sized noisy data by solving linear matrix inequalities. In this note we extend this approach to deal with unknown nonlinear polynomial systems by formulating stability certificates in the form of data-dependent sum of squares programs, whose solution directly provides a stabilizing controller and a Lyapunov function. We then derive variations of this result that lead to more advantageous controller designs. The results also reveal connections to the problem of designing a controller starting from a…
We present a framework of sequential action control (SAC) for stabilization of systems of partial differential equations which can be posed as abstract semilinear control problems in Hilbert spaces. We follow a late-lumping approach and show that the control action can be explicitly obtained from variational principles using adjoint information. Moreover, we analyze the closed-loop system obtained from the SAC feedback for the linear problem with quadratic stage costs. We apply this theory to a prototypical example of an unstable heat equation and provide numerical results as the verification …
An optimal control for a dynamical system optimizes a certain objective function. Here we consider the construction of an optimal control for a stochastic dynamical system with a random structure, Poisson perturbations and random jumps, which makes the system stable in probability. Sufficient conditions of the stability in probability are obtained, using the second Lyapunov method, in which the construction of the corresponding functions plays an important role. Here we provide a solution to the problem of optimal stabilization in a general case. For a linear system with a quadratic quality fu…
This paper studies several problems related to quadratic matrix inequalities (QMI's), i.e., inequalities in the Loewner order involving quadratic functions of matrix variables. In particular, we provide conditions under which the solution set of a QMI is nonempty, convex, bounded, or has nonempty interior. We also provide a parameterization of the solution set of a given QMI. In addition, we state results regarding the image of such sets under linear maps, which characterize a subset of ``structured" solutions to a QMI. Thereafter, we derive matrix versions of the classical S-lemma and Finsler…
In this paper, we prove the stabilizability of abstract Parabolic Integro-Differential Equations (PIDE) in a Hilbert space with decay rate $e^{-γt} $ for certain $γ> 0,$ by means of a finite dimensional controller in the feedback form. We determine a linear feedback law which is obtained by solving an algebraic Riccati equation. To prove the existence of the Riccati operator, we consider a linear quadratic optimal control problem with unbounded observation operator. The abstract theory of stabilization developed here is applied to specific problems related to viscoelastic fluids, e.g. Oldroyd …
In this work, we study model-based reinforcement learning (RL) in unknown stabilizable linear dynamical systems. When learning a dynamical system, one needs to stabilize the unknown dynamics in order to avoid system blow-ups. We propose an algorithm that certifies fast stabilization of the underlying system by effectively exploring the environment with an improved exploration strategy. We show that the proposed algorithm attains $\tilde{\mathcal{O}}(\sqrt{T})$ regret after $T$ time steps of agent-environment interaction. We also show that the regret of the proposed algorithm has only a polynom…
This paper studies uniform stabilization and social optimality for linear quadratic (LQ) mean field control problems with multiplicative noise, where agents are coupled via dynamics and individual costs. The state and control weights in cost functionals are not limited to be positive semi-definite. This leads to an indefinite LQ mean field control problem, which may still be well-posed due to deep nature of multiplicative noise. We first obtain a set of forward-backward stochastic differential equations (FBSDEs) from variational analysis, and construct a feedback control by decoupling the FBSD…
Studying structural properties of linear dynamical systems through invariant subspaces is one of the key contributions of the geometric approach to system theory. In general, a model of the dynamics is required in order to compute the invariant subspaces of interest. In this paper we overcome this limitation by finding data-driven formulas for some of the foundational tools of geometric control. In particular, for an unknown linear system, we show how controlled and conditioned invariant subspaces can be found directly from experimental data. We use our formulas and approach to (i) find a feed…
This paper considers a class of mean field linear-quadratic-Gaussian (LQG) games with model uncertainty. The drift term in the dynamics of the agents contains a common unknown function. We take a robust optimization approach where a representative agent in the limiting model views the drift uncertainty as an adversarial player. By including the mean field dynamics in an augmented state space, we solve two optimal control problems sequentially, which combined with consistent mean field approximations provides a solution to the robust game. A set of decentralized control strategies is derived by…
Control using quantized feedback is a fundamental approach to system synthesis with limited communication capacity. In this paper, we address the stabilization problem for unknown linear systems with logarithmically quantized feedback, via a direct data-driven control method. By leveraging a recently developed matrix S-lemma, we prove a sufficient and necessary condition for the existence of a common stabilizing controller for all possible dynamics consistent with data, in the form of a linear matrix inequality. Moreover, we formulate semi-definite programming to solve the coarsest quantizatio…
In this work, we are interested in tree-shaped networks of freely vibrating beams which are geometrically exact (GEB) -- in the sense that large motions (deflections, rotations) are accounted for in addition to shearing -- and linked by rigid joints. For the intrinsic GEB formulation, namely that in terms of velocities and internal forces/moments, we derive transmission conditions and show that the network is locally in time well-posed in the classical sense. Applying velocity feedback controls at the external nodes of a star-shaped network, we show by means of a quadratic Lyapunov functional …
The goal of this paper is to provide a tutorial on the so-called informativity framework for direct data-driven analysis and control. This framework achieves certified data-based analysis and control by assessing system properties and determining controllers for sets of systems unfalsified by the data. We will first introduce the informativity approach at an abstract level. Thereafter, we will report case studies where we highlight the strength of the framework in the context of various problems involving both noiseless and noisy data. In particular, we will treat controllability and stabiliza…
In standard linear quadratic (LQ) control, the first step in investigating infinite-horizon optimal control is to derive the stabilization condition with the optimal LQ controller. This paper focuses on the stabilization of an Ito stochastic system with indefinite control and state weighting matrices in the cost functional. A generalized algebraic Riccati equation (GARE) is obtained via the convergence of the generalized differential Riccati equation (GDRE) in the finite-horizon case. More importantly, the necessary and sufficient stabilization conditions for indefinite stochastic control are …
A switched equilibrium of a switched system of two subsystems is a such a point where the vector fields of the two subsystems point strictly towards one another. Using the concept of stable convex combination that was developed by Wicks-Peleties-DeCarlo (1998) for linear systems, Bolzern-Spinelli (2004) offered a design of a state feedback switching rule that is capable to stabilize an affine switched system to any switched equilibrium. The state feedback switching rule of Bolzern-Spinelli gives a nonlinear (quadratic) switching threshold passing through the switched equilibrium. In this paper…
In this paper, we consider N-level quantum angular momentum systems interacting with electromagnetic fields undergoing continuous-time measurements. We suppose unawareness of the initial state and physical parameters, entailing the introduction of an additional state representing the estimated quantum state. The evolution of the quantum state and its estimation is described by a coupled stochastic master equation. Here, we study the asymptotic behavior of such a system in presence of a feedback controller. We provide sufficient conditions on the feedback controller and on the estimated paramet…
Model-Free Output Feedback Stabilization via Policy Gradient Methods
School of Artificial Intelligence and Automation, Huazhong University of Science and Technology（人工智能与自动化学院，华中科技大学）；College of Automation, Nanjing University of Posts and Telecommunications（自动化学院，南京邮电大学）
The problem of regret minimization for online adaptive control of linear-quadratic systems is studied. In this problem, the true system transition parameters (matrices $A$ and $B$) are unknown, and the objective is to design and analyze algorithms that generate control policies with sublinear regret. Recent studies show that when the system parameters are fully unknown, there exists a choice of these parameters such that any algorithm that only uses data from the past system trajectory at best achieves a square root of time horizon regret bound, providing a hard fundamental limit on the achiev…
In this paper, we study the monitoring and control of long-term voltage stability considering load tap-changer (LTC) dynamics. We show that under generic conditions, the LTC dynamics always admit a unique stable equilibrium. For the stable equilibrium, we characterize an explicit inner approximation of its region of attraction (ROA). Compared to existing results, the computational complexity of the ROA characterization is drastically reduced. A quadratically constrained linear program formulation for the ROA characterization problem is proposed. In addition, we formulate a second-order cone pr…
We consider control systems of the type $\dot x = A x +α(t)bu$, where $u\in\R$, $(A,b)$ is a controllable pair and $α$ is an unknown time-varying signal with values in $[0,1]$ satisfying a persistent excitation condition i.e., $\int_t^{t+T}\al(s)ds\geq μ$ for every $t\geq 0$, with $0<μ\leq T$ independent on $t$. We prove that such a system is stabilizable with a linear feedback depending only on the pair $(T,μ)$ if the eigenvalues of $A$ have non-positive real part. We also show that stabilizability does not hold for arbitrary matrices $A$. Moreover, the question of whether the system can be s…
This paper deals with mathematical models of continuous crystallization described by hyperbolic systems of partial differential equations coupled with ordinary and integro-differential equations. The considered systems admit nonzero steady-state solutions with constant inputs. To stabilize these solutions, we present an approach for constructing control Lyapunov functionals based on quadratic forms in weighted L2-spaces. It is shown that the proposed control design scheme guarantees exponential stability of the closed-loop system.
The mean square stabilization problem for discrete-time networked control systems (NCSs) is investigated in this article. What the difference from most previous works is that input delay and packet losses occur simultaneously in the communication channel, moreover, the data packet dropout is modeled as a time-homogeneous Markov process which will bring some difficulties in solving the problem due to the temporal correlation. The contributions in this paper can be summarized as two points. Firstly, the equivalence condition for the solvability of linear quadratic optimal problem in finite horiz…
