This paper introduces a framework for learning a minimum-norm stabilizing controller for a system with unknown dynamics using model-free policy optimization methods. The approach begins by first designing a Control Lyapunov Function (CLF) for a (possibly inaccurate) dynamics model for the system, along with a function which specifies a minimum acceptable rate of energy dissipation for the CLF at different points in the state-space. Treating the energy dissipation condition as a constraint on the desired closed-loop behavior of the real-world system, we use penalty methods to formulate an unconstrained optimization problem over the parameters of a learned controller, which can be solved using model-free policy optimization algorithms using data collected from the plant. We discuss when the optimization learns a stabilizing controller for the real world system and derive conditions on the structure of the learned controller which ensure that the optimization is strongly convex, meaning the globally optimal solution can be found reliably. We validate the approach in simulation, first for a double pendulum, and then generalize the framework to learn stable walking controllers for underactuated bipedal robots using the Hybrid Zero Dynamics framework. By encoding a large amount of structure into the learning problem, we are able to learn stabilizing controllers for both systems with only minutes or even seconds of training data.
This paper presents a new Python software for the parametric design of stabilizing feedback laws with time delays, called Partial Pole Placement via Delay Action (P3$δ$). After an introduction recalling recent theoretical results on the multiplicity-induced-dominancy (MID) and coexisting real roots-induced-dominancy (CRRID) properties and their use for the feedback stabilization of control systems operating under time delays, the paper presents the current version of P3$δ$, which relies on the MID property to compute delayed stabilizing feedback laws for scalar differential equations with a si…
Combining a backstepping controller with a local stabilizer
Escola Politénica da Universidade de São Paulo（圣保罗大学政治工程学院）；Politecnico di Torino（都灵理工大学）；Université de Lyon（里昂大学）；Université Lyon 1（里昂第一大学）；CNRS, UMR 5007, LAGEP（法国国家科学研究中心，联合研究单位5007，实验室）；Gipsa-lab（吉普萨实验室）
For a broad class of nonlinear systems, we construct smooth control-Lyapunov functions whose derivatives along the trajectories of the systems can be made negative definite by smooth control laws that are arbitrarily small in norm. We assume our systems satisfy appropriate generalizations of the Jurdjevic-Quinn conditions. We also design state feedbacks of arbitrarily small norm that render our systems integral-input-to-state stable to actuator errors.
In this paper, we introduce a model of dynamical queue, in which the service time depends on the server utilization history. The proposed queueing model is motivated by widely accepted empirical laws describing human performance as a function of mental arousal. The objective of this paper is to design task release control policies that can stabilize the queue for the maximum possible arrival rate, assuming deterministic arrivals. First, we prove an upper bound on the maximum possible stabilizable arrival rate for any task release control policy. Then, we propose a simple threshold policy that …
In this paper we introduce a new method to design control laws for non-linear underactuated systems. Our method produces an infinite dimensional family of control laws, whereas most control techniques only produce a finite dimensional family. These control laws each come with a natural Lyapunov function. The inverted pendulum cart is used as an example. In addition, we construct an abstract system which is open loop unstable and cannot be stabilized using any linear control law, and demonstrate that our method produces a stabilizing control law.
The controller of an input-affine system is determined through minimizing a time-varying objective function, where stabilization is ensured via a Lyapunov function decay condition as constraint. This constraint is incorporated into the objective function via a barrier function. The time-varying minimum of the resulting relaxed cost function is determined by a tracking system. This system is constructed using derivatives up to second order of the relaxed cost function and improves the existing approaches in time-varying optimization. Under some mild assumptions, the tracking system yields a sol…
This paper presents a method to design a min-norm Control Lyapunov Function (CLF)-based stabilizing controller for a control-affine system with uncertain dynamics using Gaussian Process (GP) regression. In order to estimate both state and input-dependent model uncertainty, we propose a novel compound kernel that captures the control-affine nature of the problem. Furthermore, by the use of GP Upper Confidence Bound analysis, we provide probabilistic bounds of the regression error, leading to the formulation of a CLF-based stability chance constraint which can be incorporated in a min-norm optim…
Using dynamic models of tensegrity structures, we derive provable, distributed control laws for stabilizing and changing the shape of a formation of vehicles in the plane. Tensegrity models define the desired, controlled, multi-vehicle system dynamics, where each node in the tensegrity structure maps to a vehicle and each interconnecting strut or cable in the structure maps to a virtual interconnection between vehicles. Our method provides a smooth map from any desired planar formation shape to a planar tensegrity structure. The stabilizing vehicle formation shape control laws are then given b…
This paper proposes an event-triggered boundary control for the safe stabilization of the Stefan PDE system with actuator dynamics. The control law is designed by applying Zero-Order Hold (ZOH) to the continuous-time safe stabilizing controller developed in our previous work. The event-triggering mechanism is then derived so that the imposed safety conditions associated with high order Control Barrier Function (CBF) are maintained and the stability of the closed-loop system is ensured. We prove that under the proposed event-triggering mechanism, the so-called ``Zeno" behavior is always avoided…
Inverter-interfaced microgrids differ from the traditional power systems due to their lack of inertia. Vanishing timescale separation between voltage and frequency dynamics makes it critical that faster-timescale stabilizing control laws also guarantee by-construction the satisfaction of voltage limits during transients. In this article, we apply a barrier functions method to compute distributed active and reactive power setpoint control laws that certify satisfaction of voltage limits during transients. Using sum-of-squares optimization tools, we propose an algorithmic construction of these c…
In the present work, a simple algorithm for stabilizing an unknown linear time-invariant system is proposed, assuming only that this system is stabilizable. The suggested algorithm is based on first performing a partial identification of the system and then stabilizing the controllable subsystem. It should be emphasized that our approach does not depend on any prior model knowledge and requires only a minimal number of samples.
This chapter compares different formulations for Economic nonlinear Model Predictive Control (EMPC) which are all based on an established dissipativity assumption on the underlying Optimal Control Problem (OCP). This includes schemes with and without stabilizing terminal constraints, respectively, or with stabilizing terminal costs. We recall that a recently proposed approach based on gradient correcting terminal penalties implies a terminal constraint on the adjoints of the OCP. We analyze the feasibility implications of these dual/adjoint terminal constraints and we compare our findings to a…
It is well known from the seminal Brockett's theorem that the openness property of the mapping on the right-hand side of a given nonlinear ODE control system is a necessary condition for the existence of locally asymptotically stabilizing continuous stationary feedback laws. However, this condition fails to be sufficient for such a feedback stabilization. In this paper we develop an approach of variational analysis to continuous feedback stabilization of nonlinear control systems with replacing openness by the linear openness property, which has been well understood and characterized in variat…
For a control system two major issues can be considered: the stabilizability with respect to a given target, and the minimization of an integral functional (while the trajectories reach this target). Here we consider a problem where stabilizability or controllability are investigated together with the further aim of a "cost regulation", namely a state-dependent upper bounding of the functional. This paper is devoted to a crucial step in the program of establishing a chain of equivalences among degree-k stabilizability with regulated cost, asymptotic controllability with regulated cost, and the…
Three Laws of Multivariable Feedback Systems, Extended Spectral Flatness (Extended Wiener Entropy), 'Uncertainty Principles' in Variance Minimization, and Performance Limitations in Minimum Variance Estimation/Filtering
This paper develops a dissipativity-based framework for synthesis of stabilizing controllers for discrete-time nonlinear systems subject to state/input constraints. Firstly, we revisit dissipation inequalities for discrete-time nonlinear systems that involve general storage and supply functions. We prove that positive definite storage functions and cyclically negative supply functions yield asymptotic Lyapunov stability. Secondly, we define control dissipation functions (CDFs) and we construct stabilizing receding horizon controllers by minimizing CDFs subject to a cyclically negative supply c…
In this work, we address the problem of learning provably stable neural network policies for stochastic control systems. While recent work has demonstrated the feasibility of certifying given policies using martingale theory, the problem of how to learn such policies is little explored. Here, we study the effectiveness of jointly learning a policy together with a martingale certificate that proves its stability using a single learning algorithm. We observe that the joint optimization problem becomes easily stuck in local minima when starting from a randomly initialized policy. Our results sugg…
We prove a Noether-type symmetry theorem for invariant optimal control problems with unrestricted controls. The result establishes weak conservation laws along all the minimizers of the problems, including those minimizers which do not satisfy the Pontryagin Maximum Principle.
In this paper input-to-state practically stabilizing control laws for retarded, control-affine, nonlinear systems with actuator disturbance are investigated. The developed methodology is based on the Arstein's theory of control Liapunov functions and related Sontag's formula, extended to retarded systems. If the actuator disturbance is bounded, then the controller yields the solution of the closed-loop system to achieve an arbitrarily fixed neighborhood of the origin, by increasing a control tuning parameter. The considered systems can present an arbitrary number of discrete as well as distrib…
We consider the problem of making a networked system contracting by designing minimal effort local controllers. Our method combines a hierarchical contraction characterization and a matrix-balancing approach to stabilizing a Metzler matrix via minimal diagonal perturbations. We demonstrate our approach by designing local controllers that render contractive a network of FitzHugh-Nagumo neurons with a general topology of interactions.
