In this paper we consider a class of linear time invariant systems with infinitely many unstable modes. By using the parameterization of all stabilizing controllers, we show that H-infinity controllers for such systems can be computed using the techniques developed earlier for infinite dimensional plants with finitely many unstable modes. We illustrate connections between the problem solved here and an indirect method for strongly stabilizing H-infinity controller design for systems with time delays.
Weighted sensitivity minimization is studied within the framework of strongly stabilizing (stable) H-infinity controller design for a class of infinite dimensional systems. This problem has been solved by Ganesh and Pearson, [11], for finite dimensional plants using Nevanlinna-Pick interpolation. We extend their technique to a class of unstable time delay systems. Moreover, we illustrate suboptimal solutions, and their robust implementation.
The design of an observer-based state feedback (OBSF) controller with guaranteed passivity properties for port-Hamiltonian systems (PHS) is addressed using linear matrix inequalities (LMIs). The observer gain is freely chosen and the LMIs conditions such that the state feedback is equivalent to control by interconnection with an input strictly passive (ISP) and/or an output strictly passive (OSP) and zero state detectable (ZSD) port-Hamiltonian controller are established. It is shown that the proposed controller exponentially stabilizes a class of infinite-dimensional PHS and asymptotically st…
An eigenvalue based framework is developed for the stability analysis and stabilization of coupled systems with time-delays, which are naturally described by delay differential algebraic equations. The spectral properties of these equations are analyzed and their stability properties are studied, taking into account the effect of small delay perturbations. Subsequently, numerical methods for stability assessment and for designing stabilizing controllers with a prescribed structure or order, based on a direct optimization approach, are briefly addressed. The effectiveness of the approach is ill…
Various new implicit parameterizations for stabilizing controllers that allow one to impose structural constraints on the controller have been proposed lately. They are convex but infinite-dimensional, formulated in the frequency domain with no available efficient methods for computation. In this paper, we introduce a kernel version of the Youla parameterization to characterize the set of stabilizing controllers. It features a single affine constraint, which allows us to recast the controller parameterization as a novel robust filtering problem. This makes it possible to derive the first effic…
Declines in cost and concerns about the environmental impact of traditional generation have boosted the penetration of renewables and non-conventional distributed energy resources into the power grid. The intermittent availability of these resources causes the inertia of the power system to vary over time. As a result, there is a need to go beyond traditional controllers designed to regulate frequency under the assumption of invariant dynamics. This paper presents a learning-based framework for the design of stable controllers based on imitating datasets obtained from linear-quadratic regulato…
A state space based design method is given to find strongly stabilizing controllers for multiple-input-multiple-output plants (MIMO). A sufficient condition is derived for the existence of suboptimal stable H-infinity controller in terms of linear matrix inequalities (LMIs) and the controller order is twice that of the plant. A new parameterization of strongly stabilizing controllers is determined using linear fractional transformations (LFTs).
The principal task to control dynamical systems is to ensure their stability. When the system is unknown, robust approaches are promising since they aim to stabilize a large set of plausible systems simultaneously. We study linear controllers under quadratic costs model also known as linear quadratic regulators (LQR). We present two different semi-definite programs (SDP) which results in a controller that stabilizes all systems within an ellipsoid uncertainty set. We further show that the feasibility conditions of the proposed SDPs are \emph{equivalent}. Using the derived robust controller syn…
A controller synthesis method for state- and input-constrained nonlinear systems is presented that seeks continuous piecewise affine (CPA) Lyapunov-like functions and controllers simultaneously. Non-convex optimization problems are formulated on triangulated subsets of the admissible states that can be refined to meet primary control objectives, such as stability and safety, alongside secondary performance objectives. A multi-stage design is also given that enlarges the region of attraction (ROA) sequentially while allowing exclusive performance for each stage. A clear boundary for an invarian…
Self-triggered control is an improvement on event-triggered control methods. Unlike the latter, self-triggered control does not require monitoring the behavior of the system constantly. Instead, self-triggered algorithms predict the events at which the control law has to be updated before they happen, relying on system model and past information.\\ In this work, we present a self-triggered version of an event-triggered control method in which events are generated when a pseudo-Lyapunov function (PLF) associated with the system increases up to a certain limit. This approach has been shown to co…
We address the problem of model-free distributed stabilization of heterogeneous multi-agent systems using reinforcement learning (RL). Two algorithms are developed. The first algorithm solves a centralized linear quadratic regulator (LQR) problem without knowing any initial stabilizing gain in advance. The second algorithm builds upon the results of the first algorithm, and extends it to distributed stabilization of multi-agent systems with predefined interaction graphs. Rigorous proofs are provided to show that the proposed algorithms achieve guaranteed convergence if specific conditions hold…
This paper introduces a framework for learning a minimum-norm stabilizing controller for a system with unknown dynamics using model-free policy optimization methods. The approach begins by first designing a Control Lyapunov Function (CLF) for a (possibly inaccurate) dynamics model for the system, along with a function which specifies a minimum acceptable rate of energy dissipation for the CLF at different points in the state-space. Treating the energy dissipation condition as a constraint on the desired closed-loop behavior of the real-world system, we use penalty methods to formulate an uncon…
Sydney Institute for Robotics and Intelligent Systems（悉尼机器人与智能系统研究所）；Australian Centre for Field Robotics（澳大利亚野外机器人中心）；School of Aerospace, Mechanical and Mechatronic Engineering, The University of Sydney（悉尼大学航空航天、机械与机电工程学院）
This paper proposes a nonlinear policy architecture for control of partially-observed linear dynamical systems providing built-in closed-loop stability guarantees. The policy is based on a nonlinear version of the Youla parameterization, and augments a known stabilizing linear controller with a nonlinear operator from a recently developed class of dynamic neural network models called the recurrent equilibrium network (REN). We prove that RENs are universal approximators of contracting and Lipschitz nonlinear systems, and subsequently show that the the proposed Youla-REN architecture is a unive…
Stabilizing Controllers for Multi-Input, Singular Control Gain Systems
Aerospace Engineering and Engineering Mechanics, The University of Texas at Austin（德克萨斯大学奥斯汀分校航空航天工程与应用力学）；Systems and Control Engineering, Indian Institute of Technology, Bombay（印度孟买理工学院系统与控制工程）
Switching control strategies that unite a potentially high-performance but uncertified controller and a stabilizing albeit conservative controller are shown to be able to balance safety with efficiency, but have been less studied under partial observation of state. To address this gap, we propose a switching control strategy for partially observed linear-Gaussian systems with provable performance guarantees. We show that the proposed switching strategy is both safe and efficient, in the sense that: (1) the linear-quadratic cost of the system is always bounded even if the original uncertified c…
Integral reinforcement learning (IRL) was proposed in literature to obviate the requirement of drift dynamics in adaptive dynamic programming framework. Most of the online IRL schemes in literature require two sets of neural network (NNs), known as actor-critic NN and an initial stabilizing controller. Recently, for RL-based robust tracking this requirement of initial stabilizing controller and dual-approximator structure could be obviated by using a modified gradient descent-based update law containing a stabilizing term with critic-only structure. To the best of the authors' knowledge, there…
Learning Provably Stabilizing Neural Controllers for Discrete-Time Stochastic Systems
Sharif University of Technology（谢里夫理工大学）；Institute of Science and Technology Austria (ISTA)（奥地利科学技术研究所（ISTA））；Massachusetts Institute of Technology（麻省理工学院）
In this note we consider a class of linear time invariant systems with infinitely many unstable modes. By using the parameterization of all stabilizing controllers and a data transformation, we show that H-infinity controllers for such systems can be computed using the techniques developed earlier for infinite dimensional plants with finitely many unstable modes.
Stability is one of the most fundamental requirements for systems synthesis. In this paper, we address the stabilization problem for unknown linear systems via policy gradient (PG) methods. We leverage a key feature of PG for Linear Quadratic Regulator (LQR), i.e., it drives the policy away from the boundary of the unstabilizing region along the descent direction, provided with an initial policy with finite cost. To this end, we discount the LQR cost with a factor, by adaptively increasing which gradient leads the policy to the stabilizing set while maintaining a finite cost. Based on the Lyap…
This paper presents a method to design a min-norm Control Lyapunov Function (CLF)-based stabilizing controller for a control-affine system with uncertain dynamics using Gaussian Process (GP) regression. In order to estimate both state and input-dependent model uncertainty, we propose a novel compound kernel that captures the control-affine nature of the problem. Furthermore, by the use of GP Upper Confidence Bound analysis, we provide probabilistic bounds of the regression error, leading to the formulation of a CLF-based stability chance constraint which can be incorporated in a min-norm optim…
Recent advances in the reinforcement learning (RL) literature have enabled roboticists to automatically train complex policies in simulated environments. However, due to the poor sample complexity of these methods, solving RL problems using real-world data remains a challenging problem. This paper introduces a novel cost-shaping method which aims to reduce the number of samples needed to learn a stabilizing controller. The method adds a term involving a Control Lyapunov Function (CLF) -- an `energy-like' function from the model-based control literature -- to typical cost formulations. Theoreti…
Combining a backstepping controller with a local stabilizer
Escola Politénica da Universidade de São Paulo（圣保罗大学政治工程学院）；Politecnico di Torino（都灵理工大学）；Université de Lyon（里昂大学）；Université Lyon 1（里昂第一大学）；CNRS, UMR 5007, LAGEP（法国国家科学研究中心，联合研究单位5007，实验室）；Gipsa-lab（吉普萨实验室）
Recent advances in learning for control allow to synthesize vehicle controllers from learned system dynamics and maintain robust stability guarantees. However, no approach is well-suited for training linear time-invariant (LTI) controllers using arbitrary learned models of the dynamics. This article introduces a method to do so. It uses a robust control framework to derive robust stability criteria. It also uses simulated policy rollouts to obtain gradients on the controller parameters, which serve to improve the closed-loop performance. By formulating the stability criteria as penalties with …
It is known that the set of internally stabilizing controller $\mathcal{C}_{\text{stab}}$ is non-convex, but it admits convex characterizations using certain closed-loop maps: a classical result is the Youla parameterization, and two recent notions are the system-level parameterization (SLP) and the input-output parameterization (IOP). In this paper, we address the existence of new convex parameterizations and discuss potential tradeoffs of each parametrization in different scenarios. Our main contributions are: 1) We reveal that only four groups of stable closed-loop transfer matrices are equ…
This paper considers online switching control with a finite candidate controller pool, an unknown dynamical system, and unknown cost functions. The candidate controllers can be unstabilizing policies. We only require at least one candidate controller to satisfy certain stability properties, but we do not know which one is stabilizing. We design an online algorithm that guarantees finite-gain stability throughout the duration of its execution. We also provide a sublinear policy regret guarantee compared with the optimal stabilizing candidate controller. Lastly, we numerically test our algorithm…
In this paper, we study the stability of suboptimal H-infinity controllers for time-delay systems. The optimal H-infinity controller may have finitely or infinitely many unstable poles. A stable suboptimal H-infinity controller design procedure is given for each of these cases. The design methods are illustrated with examples.
The plants of nano air vehicles (NAVs) are generally unstable, adversely coupled, and uncertain. Besides, the autopilot hardware of a NAV has limited sensing and computational capabilities. Hence, these vehicles need a single controller referred to as Robust Simultaneously Stabilizing Decoupling (RSSD) output feedback controller that achieves simultaneous stabilization, desired decoupling, robustness, and performance for a finite set of unstable multi-input-multi-output adversely coupled uncertain plants. To synthesize a RSSD output feedback controller, a new method that is based on a central …
Technische Universität Chemnitz（开姆尼茨工业大学）；Skolkovo Institute of Science and Technology（斯科尔科沃科学技术研究院）
Reinforcement learning is commonly associated with training of reward-maximizing (or cost-minimizing) agents, in other words, controllers. It can be applied in model-free or model-based fashion, using a priori or online collected system data to train involved parametric architectures. In general, online reinforcement learning does not guarantee closed loop stability unless special measures are taken, for instance, through learning constraints or tailored training rules. Particularly promising are hybrids of reinforcement learning with "classical" control approaches. In this work, we suggest a …
A novel control method is proposed to ensure compatibility of safe, stabilizing control laws, i.e., simultaneous satisfaction of asymptotic stability and constraint satisfaction for nonlinear affine systems. The results are dependent on an asymptotically stabilizing control law and a zeroing control barrier function (ZCBF), but do not require a control Lyapunov function (CLF). Sufficient conditions for a region of attraction are defined for which the proposed control safely stabilizes the system, while preserving the decrescent Lyapunov function of the original stabilizing control law. The pro…
