This paper presents a method to design a min-norm Control Lyapunov Function (CLF)-based stabilizing controller for a control-affine system with uncertain dynamics using Gaussian Process (GP) regression. In order to estimate both state and input-dependent model uncertainty, we propose a novel compound kernel that captures the control-affine nature of the problem. Furthermore, by the use of GP Upper Confidence Bound analysis, we provide probabilistic bounds of the regression error, leading to the formulation of a CLF-based stability chance constraint which can be incorporated in a min-norm optimization problem. We show that this resulting optimization problem is convex, and we call it Gaussian Process-based Control Lyapunov Function Second-Order Cone Program (GP-CLF-SOCP). The data-collection process and the training of the GP regression model are carried out in an episodic learning fashion. We validate the proposed algorithm and controller in numerical simulations of an inverted pendulum and a kinematic bicycle model, resulting in stable trajectories which are very similar to the ones obtained if we actually knew the true plant dynamics.
In this paper we consider multi-agent coalitional games with uncertain value functions for which we establish distribution-free guarantees on the probability of allocation stability, i.e., agents do not have incentives to defect from the grand coalition to form subcoalitions for unseen realizations of the uncertain parameter. In case the set of stable allocations, the so called core of the game, is empty, we propose a randomized relaxation of the core. We then show that those allocations that belong to this relaxed set can be accompanied by stability guarantees in a probably approximately corr…
This paper studies the problem of utilizing data-driven adaptive control techniques to guarantee stability and safety of uncertain nonlinear systems with high relative degree. We first introduce the notion of a High Order Robust Adaptive Control Barrier Function (HO-RaCBF) as a means to compute control policies guaranteeing satisfaction of high relative degree safety constraints in the face of parametric model uncertainty. The developed approach guarantees safety by initially accounting for all possible parameter realizations but adaptively reduces uncertainty in the parameter estimates levera…
Weighted sensitivity minimization is studied within the framework of strongly stabilizing (stable) H-infinity controller design for a class of infinite dimensional systems. This problem has been solved by Ganesh and Pearson, [11], for finite dimensional plants using Nevanlinna-Pick interpolation. We extend their technique to a class of unstable time delay systems. Moreover, we illustrate suboptimal solutions, and their robust implementation.
This paper introduces a framework for learning a minimum-norm stabilizing controller for a system with unknown dynamics using model-free policy optimization methods. The approach begins by first designing a Control Lyapunov Function (CLF) for a (possibly inaccurate) dynamics model for the system, along with a function which specifies a minimum acceptable rate of energy dissipation for the CLF at different points in the state-space. Treating the energy dissipation condition as a constraint on the desired closed-loop behavior of the real-world system, we use penalty methods to formulate an uncon…
The principal task to control dynamical systems is to ensure their stability. When the system is unknown, robust approaches are promising since they aim to stabilize a large set of plausible systems simultaneously. We study linear controllers under quadratic costs model also known as linear quadratic regulators (LQR). We present two different semi-definite programs (SDP) which results in a controller that stabilizes all systems within an ellipsoid uncertainty set. We further show that the feasibility conditions of the proposed SDPs are \emph{equivalent}. Using the derived robust controller syn…
In this report, we analyze and design feedback policies for discrete-time Piecewise-Affine (PWA) systems with uncertainty in both the affine dynamics and the polytopic partition. The main idea is to utilise the Difference-of-Convex (DC) decomposition of continuous PWA systems to derive quadratic Lyapunov functions as stability certificates and stabilizing affine policies in a higher dimensional space. When projected back to the state space, we obtain time-varying PWQ Lyapunov functions and time-varying PWA feedback policies.
Recent studies have demonstrated the potential of flexible loads in providing frequency response services. However, uncertainty and variability in various weather-related and end-use behavioral factors often affect the demand-side control performance. This work addresses this problem with the design of a demand-side control to achieve frequency response under load uncertainties. Our approach involves modeling the load uncertainties via stochastic processes that appear as both multiplicative and additive to the system states in closed-loop power system dynamics. Extending the recently developed…
The controller of an input-affine system is determined through minimizing a time-varying objective function, where stabilization is ensured via a Lyapunov function decay condition as constraint. This constraint is incorporated into the objective function via a barrier function. The time-varying minimum of the resulting relaxed cost function is determined by a tracking system. This system is constructed using derivatives up to second order of the relaxed cost function and improves the existing approaches in time-varying optimization. Under some mild assumptions, the tracking system yields a sol…
École Polytechnique Fédérale de Lausanne（洛桑联邦理工学院）
We address the problem of designing stabilizing control policies for nonlinear systems in discrete-time, while minimizing an arbitrary cost function. When the system is linear and the cost is convex, the System Level Synthesis (SLS) approach offers an effective solution based on convex programming. Beyond this case, a globally optimal solution cannot be found in a tractable way, in general. In this paper, we develop a parametrization of all and only the control policies stabilizing a given time-varying nonlinear system in terms of the combined effect of 1) a strongly stabilizing base controlle…
This paper proposes an event-triggered boundary control for the safe stabilization of the Stefan PDE system with actuator dynamics. The control law is designed by applying Zero-Order Hold (ZOH) to the continuous-time safe stabilizing controller developed in our previous work. The event-triggering mechanism is then derived so that the imposed safety conditions associated with high order Control Barrier Function (CBF) are maintained and the stability of the closed-loop system is ensured. We prove that under the proposed event-triggering mechanism, the so-called ``Zeno" behavior is always avoided…
This paper studies a discrete-time stochastic control problem with linear quadratic criteria over an infinite-time horizon. We focus on a class of control systems whose system matrices are associated with random parameters involving unknown statistical properties. In particular, we design a distributed Q-learning algorithm to tackle the Riccati equation and derive the optimal controller stabilizing the system. The key technique is that we convert the problem of solving the Riccati equation into deriving the zero point of a matrix equation and devise a distributed stochastic approximation metho…
The plants of nano air vehicles (NAVs) are generally unstable, adversely coupled, and uncertain. Besides, the autopilot hardware of a NAV has limited sensing and computational capabilities. Hence, these vehicles need a single controller referred to as Robust Simultaneously Stabilizing Decoupling (RSSD) output feedback controller that achieves simultaneous stabilization, desired decoupling, robustness, and performance for a finite set of unstable multi-input-multi-output adversely coupled uncertain plants. To synthesize a RSSD output feedback controller, a new method that is based on a central …
This chapter compares different formulations for Economic nonlinear Model Predictive Control (EMPC) which are all based on an established dissipativity assumption on the underlying Optimal Control Problem (OCP). This includes schemes with and without stabilizing terminal constraints, respectively, or with stabilizing terminal costs. We recall that a recently proposed approach based on gradient correcting terminal penalties implies a terminal constraint on the adjoints of the OCP. We analyze the feasibility implications of these dual/adjoint terminal constraints and we compare our findings to a…
The design of robust orbitally stabilizing feedback is considered. From a known orbitally stabilizing controller for a nominal, disturbance-free system, a robustifying feedback extension is designed utilizing the sliding-mode control (SMC) methodology. The main contribution of the paper is to provide a constructive procedure for designing the time-invariant switching function used in the SMC synthesis. More specifically, its zero-level set (the sliding manifold) is designed using a real Floquet-Lyapunov transformation to locally correspond to an invariant subspace of the Monodromy matrix of a …
In this article, we focus on the global stabilizability problem for a class of second order uncertain stochastic control systems, where both the drift term and the diffusion term are nonlinear functions of the state variables and the control variables. We will show that the widely applied proportional-derivative(PD) control in engineering practice has the ability to globally stabilize such systems in the mean square sense, provided that the upper bounds of the partial derivatives of the nonlinear functions satisfy a certain algebraic inequality. It will also be proved that the stabilizing PD p…
In large-scale networks of uncertain dynamical systems, where communication is limited and there is a strong interaction among subsystems, learning local models and control policies offers great potential for designing high-performance controllers. At the same time, the lack of safety guarantees, here considered in the form of constraint satisfaction, prevents the use of data-driven techniques to safety-critical distributed systems. This paper presents a safety framework that guarantees constraint satisfaction for uncertain distributed systems while learning. The framework considers linear sys…
Is Linear Feedback on Smoothed Dynamics Sufficient for Stabilizing Contact-Rich Plans?
University of California, Los Angeles（加州大学洛杉矶分校）；RAI Institute (formerly Boston Dynamics AI Institute)（RAI研究院）；MIT CSAIL（麻省理工学院CSAIL）；New York University（纽约大学）；Georgia Tech（佐治亚理工学院）
The H2 guaranteed cost decentralized control problem is investigated in this work. More specifically, on the basis of an appropriate H2 re-formulation that we put in place, the optimal control problem in the presence of parameter uncertainties is then suitably characterized by convex restriction and solved in parameter space. It is shown that a set of stabilizing decentralized controller gains for the uncertain system is parameterized in a convex set through appropriate convex restriction, and then an approximated conic optimization problem is constructed. This facilitates the use of the symme…
MSACL: Multi-Step Actor-Critic Learning with Lyapunov Certificates for Exponentially Stabilizing Control
School of Mathematical Sciences, Beihang University（北京航空航天大学数学科学学院）；School of Automation Science and Electrical Engineering, Beihang University（北京航空航天大学自动化科学与电气工程学院）；Fujian Key Laboratory of Financial Information Processing, Putian University（福建金融信息处理重点实验室）
Robust imitation learning seeks to mimic expert controller behavior while ensuring stability, but current methods require accurate plant models. Here, robust imitation learning is addressed for stabilizing poorly modeled plants with linear dynamic output feedback. Open-loop input-output properties are used to characterize an uncertain plant, and the feedback matrix of the dynamic controller is learned while enforcing stability through the controller's open-loop QSR-dissipativity properties. The imitation learning method is applied to two systems with parametric uncertainty.
Technische Universität Chemnitz（开姆尼茨工业大学）；Skolkovo Institute of Science and Technology（斯科尔科沃科学技术研究院）
Reinforcement learning is commonly associated with training of reward-maximizing (or cost-minimizing) agents, in other words, controllers. It can be applied in model-free or model-based fashion, using a priori or online collected system data to train involved parametric architectures. In general, online reinforcement learning does not guarantee closed loop stability unless special measures are taken, for instance, through learning constraints or tailored training rules. Particularly promising are hybrids of reinforcement learning with "classical" control approaches. In this work, we suggest a …
Optimal control is an essential tool for stabilizing complex nonlinear systems. However, despite the extensive impacts of methods such as receding horizon control, dynamic programming and reinforcement learning, the design of cost functions for a particular system often remains a heuristic-driven process of trial and error. In this paper we seek to gain insights into how the choice of cost function interacts with the underlying structure of the control system and impacts the amount of computation required to obtain a stabilizing controller. We treat the cost design problem as a two-step proces…
We consider the problem of stabilization of a linear system, under state and control constraints, and subject to bounded disturbances and unknown parameters in the state matrix. First, using a simple least square solution and available noisy measurements, the set of admissible values for parameters is evaluated. Second, for the estimated set of parameter values and the corresponding linear interval model of the system, two interval predictors are recalled and an unconstrained stabilizing control is designed that uses the predicted intervals. Third, to guarantee the robust constraint satisfacti…
Stabilizing controller design and region of attraction (RoA) estimation are essential in nonlinear control. Moreover, it is challenging to implement a control Lyapunov function (CLF) in practice when only partial knowledge of the system is available. We propose a learning framework that can synthesize state-feedback controllers and a CLF for control-affine nonlinear systems with unstructured uncertainties. Based on a regularity condition on these uncertainties, we model them as bounded disturbances and prove that a CLF for the nominal system (estimate of the true system) is an input-to-state s…
In this paper, the notion of robust strict QSR-dissipativity is applied to solve the static output feedback control problem for a class of continuous-time nonlinear rational systems subject to input saturation and bounded parametric uncertainties. A local dissipativity condition is combined with generalized sector conditions to formulate the synthesis of a stabilizing controller in terms of linear matrix inequalities. The strategy applies to general static output feedback design without any restrictions on the plant output equation. An iterative algorithm based on linear matrix inequalities is…
Self-triggered control is an improvement on event-triggered control methods. Unlike the latter, self-triggered control does not require monitoring the behavior of the system constantly. Instead, self-triggered algorithms predict the events at which the control law has to be updated before they happen, relying on system model and past information.\\ In this work, we present a self-triggered version of an event-triggered control method in which events are generated when a pseudo-Lyapunov function (PLF) associated with the system increases up to a certain limit. This approach has been shown to co…
A controller synthesis method for state- and input-constrained nonlinear systems is presented that seeks continuous piecewise affine (CPA) Lyapunov-like functions and controllers simultaneously. Non-convex optimization problems are formulated on triangulated subsets of the admissible states that can be refined to meet primary control objectives, such as stability and safety, alongside secondary performance objectives. A multi-stage design is also given that enlarges the region of attraction (ROA) sequentially while allowing exclusive performance for each stage. A clear boundary for an invarian…
Recent advances in the reinforcement learning (RL) literature have enabled roboticists to automatically train complex policies in simulated environments. However, due to the poor sample complexity of these methods, solving RL problems using real-world data remains a challenging problem. This paper introduces a novel cost-shaping method which aims to reduce the number of samples needed to learn a stabilizing controller. The method adds a term involving a Control Lyapunov Function (CLF) -- an `energy-like' function from the model-based control literature -- to typical cost formulations. Theoreti…
Adaptive Control Lyapunov Functions (aCLFs) were introduced 20 years ago, and provided a Lyapunov-based methodology for stabilizing systems with parameter uncertainty. The goal of this paper is to revisit this classic formulation in the context of safety-critical control. This will motivate a variant of aCLFs in the context of safety: adaptive Control Barrier Functions (aCBFs). Our proposed approach adaptively achieves safety by keeping the systems state within a safe set even in the presence of parametric model uncertainty. We unify aCLFs and aCBFs into a single control methodology for system…
First and High Order Sliding Mode-Multimodel Stabilizing Control Synthesis using Single and Several Sliding Surfaces for Nonlinear Systems: Simulation on an Autonomous Underwater Vehicles (AUV)
Ecole Polytechnique de Tunisie（突尼斯高等理工学院）；L.S.A Laboratoire des Systèmes Avancés（先进系统实验室）
