Learning-based control of linear systems received a lot of attentions recently. In popular settings, the true dynamical models are unknown to the decision-maker and need to be interactively learned by applying control inputs to the systems. Unlike the matured literature of efficient reinforcement learning policies for adaptive control of a single system, results on joint learning of multiple systems are not currently available. Especially, the important problem of fast and reliable joint-stabilization remains unaddressed and so is the focus of this work. We propose a novel joint learning-based stabilization algorithm for quickly learning stabilizing policies for all systems understudy, from the data of unstable state trajectories. The presented procedure is shown to be notably effective such that it stabilizes the family of dynamical systems in an extremely short time period.
Model-Free Output Feedback Stabilization via Policy Gradient Methods
School of Artificial Intelligence and Automation, Huazhong University of Science and Technology（人工智能与自动化学院，华中科技大学）；College of Automation, Nanjing University of Posts and Telecommunications（自动化学院，南京邮电大学）
In this work, we study model-based reinforcement learning (RL) in unknown stabilizable linear dynamical systems. When learning a dynamical system, one needs to stabilize the unknown dynamics in order to avoid system blow-ups. We propose an algorithm that certifies fast stabilization of the underlying system by effectively exploring the environment with an improved exploration strategy. We show that the proposed algorithm attains $\tilde{\mathcal{O}}(\sqrt{T})$ regret after $T$ time steps of agent-environment interaction. We also show that the regret of the proposed algorithm has only a polynom…
California Institute of Technology（加州理工学院）；University of California San Diego（加州大学圣地亚哥分校）；Carnegie Mellon University（卡内基梅隆大学）；Purdue University（普渡大学）
Learning a dynamical system requires stabilizing the unknown dynamics to avoid state blow-ups. However, current reinforcement learning (RL) methods lack stabilization guarantees, which limits their applicability for the control of safety-critical systems. We propose a model-based RL framework with formal stability guarantees, Krasovskii Constrained RL (KCRL), that adopts Krasovskii's family of Lyapunov functions as a stability constraint. The proposed method learns the system dynamics up to a confidence interval using feature representation, e.g. Random Fourier Features. It then solves a const…
In the present work, a simple algorithm for stabilizing an unknown linear time-invariant system is proposed, assuming only that this system is stabilizable. The suggested algorithm is based on first performing a partial identification of the system and then stabilizing the controllable subsystem. It should be emphasized that our approach does not depend on any prior model knowledge and requires only a minimal number of samples.
We investigate the problem of stabilizing an unknown networked linear system under communication constraints and adversarial disturbances. We propose the first provably stabilizing algorithm for the problem. The algorithm uses a distributed version of nested convex body chasing to maintain a consistent estimate of the network dynamics and applies system level synthesis to determine a distributed controller based on this estimated model. Our approach avoids the need for system identification and accommodates a broad class of communication delay while being fully distributed and scaling favorabl…
Motivated by the goal of having a building block in the direct design of data-driven controllers for nonlinear systems, we show how, for an unknown discrete-time bilinear system, the data collected in an offline open-loop experiment enable us to design a feedback controller and provide a guaranteed under-approximation of its basin of attraction. Both can be obtained by solving a linear matrix inequality for a fixed scalar parameter, and possibly iterating on different values of that parameter. The results of this data-based approach are compared with the ideal case when the model is known perf…
Adaptively controlling and minimizing regret in unknown dynamical systems while controlling the growth of the system state is crucial in real-world applications. In this work, we study the problem of stabilization and regret minimization of linear over-actuated dynamical systems. We propose an optimism-based algorithm that leverages possibility of switching between actuating modes in order to alleviate state explosion during initial time steps. We theoretically study the rate at which our algorithm learns a stabilizing controller and prove that it achieves a regret upper bound of $\mathcal{O}(…
A design framework recently has been developed to stabilize interconnected multiagent systems in a distributed manner, and systematically capture the architectural aspect of cyber-physical systems. Such a control theoretic framework, however, results in a stabilization protocol which is passive with respect to the cyber attacks and conservative regarding the guaranteed level of resiliency. We treat the control layer topology and stabilization gains as the degrees of freedom, and develop a mixed control and cybersecurity design framework to address the above concerns. From a control perspective…
Linear dynamical systems are canonical models for learning-based control of plants with uncertain dynamics. The setting consists of a stochastic differential equation that captures the state evolution of the plant understudy, while the true dynamics matrices are unknown and need to be learned from the observed data of state trajectory. An important issue is to ensure that the system is stabilized and destabilizing control actions due to model uncertainties are precluded as soon as possible. A reliable stabilization procedure for this purpose that can effectively learn from unstable data to sta…
In a recent paper we have shown how to learn controllers for unknown linear systems using finite-sized noisy data by solving linear matrix inequalities. In this note we extend this approach to deal with unknown nonlinear polynomial systems by formulating stability certificates in the form of data-dependent sum of squares programs, whose solution directly provides a stabilizing controller and a Lyapunov function. We then derive variations of this result that lead to more advantageous controller designs. The results also reveal connections to the problem of designing a controller starting from a…
Studying structural properties of linear dynamical systems through invariant subspaces is one of the key contributions of the geometric approach to system theory. In general, a model of the dynamics is required in order to compute the invariant subspaces of interest. In this paper we overcome this limitation by finding data-driven formulas for some of the foundational tools of geometric control. In particular, for an unknown linear system, we show how controlled and conditioned invariant subspaces can be found directly from experimental data. We use our formulas and approach to (i) find a feed…
For a parameter-unknown linear descriptor system, this paper proposes data-driven methods to testify the system's type and controllability and then to stabilize it. First, a data-based condition is developed to identify whether this unknown system is a descriptor system or is equivalent to a normal system. Furthermore, various controllability concepts are testified by replacing the descriptor system's matrices with data. Finally, a data-based decomposing method is proposed to transfer the nominal system into its slow-fast subsystems' form, so that a state feedback controller for the slow subsy…
Control using quantized feedback is a fundamental approach to system synthesis with limited communication capacity. In this paper, we address the stabilization problem for unknown linear systems with logarithmically quantized feedback, via a direct data-driven control method. By leveraging a recently developed matrix S-lemma, we prove a sufficient and necessary condition for the existence of a common stabilizing controller for all possible dynamics consistent with data, in the form of a linear matrix inequality. Moreover, we formulate semi-definite programming to solve the coarsest quantizatio…
Finite-Time Stabilization of Longitudinal Control for Autonomous Vehicles via a Model-Free Approach
LIX (CNRS, UMR 7161), \'Ecole polytechnique, 91128 Palaiseau, France (e-mail: )；Centre de Recherche STIC, Universit\'e de Reims Champagne-Ardenne, IUT de Troyes, 9 rue du Qu\'ebec, 10000 Troyes, France (e-mail: )
We study the self-triggered stabilization of discrete-time linear systems with quantized state measurements. In the networked control system we consider, sensors may be spatially distributed and be connected to a self-triggering mechanism through finite data-rate channels. Each sensor independently encodes its measurements and sends them to the self-triggering mechanism. The self-triggering mechanism integrates quantized measurement data and then computes sampling times. Assuming that the closed-loop system is stable in the absence of quantization and self-triggered sampling, we propose a join…
Dirty derivatives are routinely used in industrial settings, particularly in the implementation of the derivative term in PID control, and are especially appealing due to their noise-attenuation and model-free characteristics. In this paper, we provide a Lyapunov-based proof for the stability of linear time-invariant control systems in controller canonical form when utilizing dirty derivatives in place of observers for the purpose of output feedback. This is, to the best of the authors' knowledge, the first time that stability proofs are provided for the use of dirty derivatives in lieu of der…
It is an interesting open problem to achieve adaptive prescribed-time control for strict-feedback systems with unknown and fast or even abrupt time-varying parameters. In this paper we present a solution with the aid of several design and analysis innovations. First, by using a spatiotemporal transformation, we convert the original system operational over finite time interval into one operational over infinite time interval, allowing for Lyapunov asymptotic design and recasting prescribed-time stabilization on finite time domain into asymptotic stabilization on infinite time domain. Second, to…
In this paper, adaptive prescribed finite time stabilization of uncertain single-input and single-output nonlinear systems is considered in the presence of unknown states, unknown parameters, external load disturbance, and non-symmetric input saturation. A prescribed finite time disturbance observer is designed to approximate the unmeasured external disturbance. Also, a nonsingular prescribed finite time terminal sliding mode control is proposed for the closed-loop control of the system with the non-symmetric input saturation. Extended Kalman filter algorithm is employed for the real-time esti…
A computationally efficient method for online joint state inference and dynamical model learning is presented. The dynamical model combines an a priori known, physically derived, state-space model with a radial basis function expansion representing unknown system dynamics and inherits properties from both physical and data-driven modeling. The method uses an extended Kalman filter approach to jointly estimate the state of the system and learn the unknown system dynamics, via the parameters of the basis function expansion. The key contribution is a computational complexity reduction compared to…
Optimal Universal Controllers for Roll Stabilization
Oceaneering AGV Systems B.V.（Oceaneering AGV系统有限公司）；University of Groningen（格罗宁根大学）；Politecnico di Torino（都灵理工大学）；Institute for Problems of Mechanical Engineering（机械工程学院问题研究所）
We present a method for incremental modeling and time-varying control of unknown nonlinear systems. The method combines elements of evolving intelligence, granular machine learning, and multi-variable control. We propose a State-Space Fuzzy-set-Based evolving Modeling (SS-FBeM) approach. The resulting fuzzy model is structurally and parametrically developed from a data stream with focus on memory and data coverage. The fuzzy controller also evolves, based on the data instances and fuzzy model parameters. Its local gains are redesigned in real-time -- whenever the corresponding local fuzzy mode…
How to practically maintain the frequency stability of large-scale power systems by a decentralized way is a simple but non-trivial question. In other words, is it possible to design any local controller without understanding the other controlled areas and with less understanding of network structure? With respect to the special properties of physical interaction between the local areas, this paper suggests two existing theories for tackling this issue. Firstly, passivity theory is shown to be a candidate for frequency control problem using swing equation. Based on the passivity of swing dynam…
It is nontrivial to achieve exponential stability even for time-invariant nonlinear systems with matched uncertainties and persistent excitation (PE) condition. In this paper, without the need for PE condition, we address the problem of global exponential stabilization of strict-feedback systems with mismatched uncertainties and unknown yet time-varying control gains. The resultant control, embedded with time-varying feedback gains, is capable of ensuring global exponential stability of parametric-strict-feedback systems in the absence of persistence of excitation. By using the enhanced Nussba…
SiliconSynapse Laboratory at Northeastern University（东北大学硅突触实验室）
Our goal in this work is to expand the theory and practice of robot locomotion by addressing critical challenges associated with the robotic biomimicry of bat aerial locomotion. Bats wings exhibit fast wing articulation and can mobilize as many as 40 joints within a single wingbeat. Mimicking bat flight can be a significant ordeal and the current design paradigms have failed as they assume only closed-loop feedback roles through sensors and conventional actuators while ignoring the computational role carried by morphology. In this paper, we propose a design framework called Morphing via Integr…
In this paper, we consider a class of denial-of-service (DoS) attacks, which aims at overloading the communication channel. On top of the security issue, continuous or periodic transmission of information within feedback loop is necessary for the effective control and stabilization of the system. In addition, uncertainty---originating from variation of parameters or unmodeled system dynamics---plays a key role in the system's stability. To address these three critical factors, we solve the joint control and security problem for an uncertain discrete-time Networked Control System (NCS) subject …
Predefined-Time Resilient Integral Reinforcement Learning for Input-Constrained Unknown Nonlinear Systems Under FDI Attacks and Disturbances: A Fully Data-Driven Approach
Faculty of Mechanical Engineering, Ho Chi Minh City University of Technology (HCMUT), Vietnam National University Ho Chi Minh City (VNU-HCM)（胡志明市工业大学机械工程学院，越南国立大学胡志明市分校）
