Learning and synthesizing stabilizing controllers for unknown nonlinear control systems is a challenging problem for real-world and industrial applications. Koopman operator theory allows one to analyze nonlinear systems through the lens of linear systems and nonlinear control systems through the lens of bilinear control systems. The key idea of these methods lies in the transformation of the coordinates of the nonlinear system into the Koopman observables, which are coordinates that allow the representation of the original system (control system) as a higher dimensional linear (bilinear control) system. However, for nonlinear control systems, the bilinear control model obtained by applying Koopman operator based learning methods is not necessarily stabilizable. Simultaneous identification of stabilizable lifted bilinear control systems as well as the associated Koopman observables is still an open problem. In this paper, we propose a framework to construct these stabilizable bilinear models and identify its associated observables from data by simultaneously learning a bilinear Koopman embedding for the underlying unknown control affine nonlinear system as well as a Control Lyapunov Function (CLF) for the Koopman based bilinear model using a learner and falsifier. Our proposed approach thereby provides provable guarantees of asymptotic stability for the Koopman based representation of the unknown control affine nonlinear control system as a bilinear system. Numerical simulations are provided to validate the efficacy of our proposed class of stabilizing feedback controllers for unknown control-affine nonlinear systems.
Koopman spectral theory has grown in the past decade as a powerful tool for dynamical systems analysis and control. In this paper, we show how recent data-driven techniques for estimating Koopman-Invariant subspaces with neural networks can be leveraged to extract Lyapunov certificates for the underlying system. In our work, we specifically focus on systems with a limit-cycle, beyond just an isolated equilibrium point, and use Koopman eigenfunctions to efficiently parameterize candidate Lyapunov functions to construct forward-invariant sets under some (unknown) attractor dynamics. Additionally…
Koopman operator theory enables a global linear representation of a given nonlinear dynamical system by transforming the nonlinear dynamics into a higher dimensional observable function space where the evolution of observable functions is governed by an infinite-dimensional linear operator. For practical application of Koopman operator theory, various data-driven methods have been developed to derive lifted state-space models via approximation to the Koopman operator. Based on approximate models, several Koopman-based model predictive control (KMPC) schemes have been proposed. However, since a…
In this work, a predictive control framework is presented for feedback stabilization of nonlinear systems. To achieve this, we integrate Koopman operator theory with Lyapunov-based model predictive control (LMPC). The main idea is to transform nonlinear dynamics from state-space to function space using Koopman eigenfunctions - for control affine systems this results in a bilinear model in the (lifted) function space. Then, a predictive controller is formulated in Koopman eigenfunction coordinates which uses an auxiliary Control Lyapunov Function (CLF) based bounded controller as a constraint t…
Deep Neural Koopman Operator-based Economic Model Predictive Control of Shipboard Carbon Capture System
Nanyang Technological University（南洋理工大学）；School of Chemistry, Chemical Engineering and Biotechnology（化学、化学工程与生物技术学院）；Nanyang Environment & Water Research Institute (NEWRI)（南洋环境与水研究所）
We consider the problem of synthesis of safe controllers for nonlinear systems with unknown dynamics using Control Barrier Functions (CBF). We utilize Koopman operator theory (KOT) to associate the (unknown) nonlinear system with a higher dimensional bilinear system and propose a data-driven learning framework that uses a learner and a falsifier to simultaneously learn the Koopman operator based bilinear system and a corresponding CBF. We prove that the learned CBF for the latter bilinear system is also a valid CBF for the unknown nonlinear system by characterizing the $\ell^2$-norm error boun…
CKNet: A Convolutional Neural Network Based on Koopman Operator for Modeling Latent Dynamics from Pixels
College of Intelligence Science and Technology, National University of Defense Technology（国防科技大学智能科学与技术学院）；Unmanned System Technology Research Center, NIIDT（NIIDT无人系统技术研究中心）
In recent years data-driven analysis of dynamical systems has attracted a lot of attention and transfer operator techniques, namely, Perron-Frobenius and Koopman operators are being used almost ubiquitously. Since data is always obtained in discrete-time, in this paper, we propose a purely data-driven approach for the design of a stabilizing feedback control law for a general class of discrete-time control-affine non-linear systems. In particular, we use the Koopman operator to lift a control-affine system to a higher-dimensional space, where the control system's evolution is bilinear. We anal…
We develop a data-driven, model-free approach for the optimal control of the dynamical system. The proposed approach relies on the Deep Neural Network (DNN) based learning of Koopman operator for the purpose of control. In particular, DNN is employed for the data-driven identification of basis function used in the linear lifting of nonlinear control system dynamics. The controller synthesis is purely data-driven and does not rely on a priori domain knowledge. The OpenAI Gym environment, employed for Reinforcement Learning-based control design, is used for data generation and learning of Koopma…
Electrical neuromodulation as a palliative treatment has been increasingly used in the control of epilepsy. However, current neuromodulations commonly implement predetermined actuation strategies and lack the capability of self-adaptively adjusting stimulation inputs. In this work, rooted in optimal control theory, we propose a Koopman-MPC framework for real-time closed-loop electrical neuromodulation in epilepsy, which integrates i) a deep Koopman operator based dynamical model to predict the temporal evolution of epileptic EEG with an approximate finite-dimensional linear dynamics and ii) a …
NTT Communication Science Laboratories（NTT通信科学实验室）；Institute of Mathematics for Industry, Kyushu University（九州大学数学工学研究所）；Center for Advanced Intelligence Project, RIKEN（理化学研究所高级智能项目中心）
We propose a data-driven method for controlling the frequency and convergence rate of black-box nonlinear dynamical systems based on the Koopman operator theory. With the proposed method, a policy network is trained such that the eigenvalues of a Koopman operator of controlled dynamics are close to the target eigenvalues. The policy network consists of a neural network to find a Koopman invariant subspace, and a pole placement module to adjust the eigenvalues of the Koopman operator. Since the policy network is differentiable, we can train it in an end-to-end fashion using reinforcement learni…
We propose new methods for learning control policies and neural network Lyapunov functions for nonlinear control problems, with provable guarantee of stability. The framework consists of a learner that attempts to find the control and Lyapunov functions, and a falsifier that finds counterexamples to quickly guide the learner towards solutions. The procedure terminates when no counterexample is found by the falsifier, in which case the controlled nonlinear system is provably stable. The approach significantly simplifies the process of Lyapunov control design, provides end-to-end correctness gua…
Learning for control of dynamical systems with formal guarantees remains a challenging task. This paper proposes a learning framework to simultaneously stabilize an unknown nonlinear system with a neural controller and learn a neural Lyapunov function to certify a region of attraction (ROA) for the closed-loop system. The algorithmic structure consists of two neural networks and a satisfiability modulo theories (SMT) solver. The first neural network is responsible for learning the unknown dynamics. The second neural network aims to identify a valid Lyapunov function and a provably stabilizing …
The lack of stability guarantee restricts the practical use of learning-based methods in core control problems in robotics. We develop new methods for learning neural control policies and neural Lyapunov critic functions in the model-free reinforcement learning (RL) setting. We use sample-based approaches and the Almost Lyapunov function conditions to estimate the region of attraction and invariance properties through the learned Lyapunov critic functions. The methods enhance stability of neural controllers for various nonlinear systems including automobile and quadrotor control.
University of California, Santa Barbara（加州大学圣巴巴拉分校）
Koopman operators model nonlinear dynamics as a linear dynamic system acting on a nonlinear function as the state. This nonstandard state is often called a Koopman observable and is usually approximated numerically by a superposition of functions drawn from a \textit{dictionary}. A widely used algorithm, is \textit{Extended Dynamic Mode Decomposition}, where the dictionary functions are drawn from a fixed, homogeneous class of functions. Recently, deep learning combined with EDMD has been used to learn novel dictionary functions in an algorithm called deep dynamic mode decomposition (deepDMD).…
University of California, Santa Barbara（加州大学圣巴巴拉分校）
Koopman operators model nonlinear dynamics as a linear dynamic system acting on a nonlinear function as the state. This nonstandard state is often called a Koopman observable and is usually approximated numerically by a superposition of functions drawn from a dictionary. In a widely used algorithm, Extended Dynamic Mode Decomposition, the dictionary functions are drawn from a fixed class of functions. Recently, deep learning combined with EDMD has been used to learn novel dictionary functions in an algorithm called deep dynamic mode decomposition (deepDMD). The learned representation both (1) …
Sparked by the Willems' fundamental lemma, a class of data-driven control methods has been developed for LTI systems. At the same time, the Koopman operator theory attempts to cast a nonlinear control problem into a standard linear one albeit infinite-dimensional. Motivated by these two ideas, a data-driven control scheme for nonlinear systems is proposed in this work. The proposed scheme is compatible with most differential regressors enabling offline learning. In particular, the model uncertainty is considered, enabling a novel data-driven simulation framework based on Wasserstein distance. …
Deep Koopman Economic Model Predictive Control of a Pasteurisation Unit
Slovak University of Technology in Bratislava, Institute of Information Engineering, Automation, and Mathematics（斯洛伐克布拉迪斯拉发工业大学，信息工程、自动化与数学研究所）；Czech Technical University, Department of Control Engineering（捷克技术大学，控制工程系）
Toyota Research Institute（丰田研究院）；Stanford University（斯坦福大学）；Massachusetts Institute of Technology（麻省理工学院）
Deep learning has had a far reaching impact in robotics. Specifically, deep reinforcement learning algorithms have been highly effective in synthesizing neural-network controllers for a wide range of tasks. However, despite this empirical success, these controllers still lack theoretical guarantees on their performance, such as Lyapunov stability (i.e., all trajectories of the closed-loop system are guaranteed to converge to a goal state under the control policy). This is in stark contrast to traditional model-based controller design, where principled approaches (like LQR) can synthesize stabl…
Generalized Momenta-Based Koopman Formalism for Robust Control of Euler-Lagrangian Systems
Department of Mechanical Engineering, Indian Institute of Science（机械工程系，印度科学研究院）；Robert Bosch Centre for Cyber-Physical Systems, Indian Institute of Science（网络物理系统研究中心，印度科学研究院）
Continuous monitoring of the spatio-temporal dynamic behavior of critical infrastructure networks, such as the power systems, is a challenging but important task. In particular, accurate and timely prediction of the (electro-mechanical) transient dynamic trajectories of the power grid is necessary for early detection of any instability and prevention of catastrophic failures. Existing approaches for the prediction of dynamic trajectories either rely on the availability of accurate physical models of the system, use computationally expensive time-domain simulations, or are applicable only at lo…
This paper presents a Koopman lifting linearization method that is applicable to nonlinear dynamical systems having both stable and unstable regions. It is known that DMD and other standard data-driven methods face a fundamental difficulty in constructing a Koopman model when applied to unstable systems. Here we solve the problem by incorporating knowledge about a nonlinear state equation with a learning method for finding an effective set of observables. In a lifted space, stable and unstable regions are separated into independent subspaces. Based on this property, we propose to find effectiv…
This paper proposes a novel transient stability assessment tool for networked microgrids based on neural Lyapunov methods. Assessing transient stability is formulated as a problem of estimating the dynamic security region of networked microgrids. We leverage neural networks to learn a local Lyapunov function in the state space. The largest security region is estimated based on the learned neural Lyapunov function, and it is used for characterizing disturbances that the networked microgrids can tolerate. The proposed method is tested and validated in a grid-connected microgrid, three networked …
