Technical University of Munich（慕尼黑工业大学）；Aalborg University（奥尔堡大学）
Petar Bevanda, Max Beier, Shahab Heshmati-Alamdari, Stefan Sosnowski, Sandra Hirche
备注：
Final version, accepted for presentation at the 6th IFAC Conference on Intelligent Control and Automation Sciences (ICONS), 2022. arXiv admin note: text overlap with arXiv:2112.04085
We propose a novel framework for learning linear time-invariant (LTI) models for a class of continuous-time non-autonomous nonlinear dynamics based on a representation of Koopman operators. In general, the operator is infinite-dimensional but, crucially, linear. To utilize it for efficient LTI control design, we learn a finite representation of the Koopman operator that is linear in controls while concurrently learning meaningful lifting coordinates. For the latter, we rely on Koopmanizing Flows - a diffeomorphism-based representation of Koopman operators and extend it to systems with linear control entry. With such a learned model, we can replace the nonlinear optimal control problem with quadratic cost to that of a linear quadratic regulator (LQR), facilitating efficacious optimal control for nonlinear systems. The superior control performance of the proposed method is demonstrated on simulation examples.
Stability-Certified On-Policy Data-Driven LQR via Recursive Learning and Policy Gradient
Department of Electrical, Electronic and Information Engineering, Alma Mater Studiorum - Università di Bologna（阿尔玛·mater studiorum - 博洛尼亚大学电气、电子与信息工程系）
We present a data-driven method for solving the linear quadratic regulator problem for systems with multiplicative disturbances, the distribution of which is only known through sample estimates. We adopt a distributionally robust approach to cast the controller synthesis problem as semidefinite programs. Using results from high dimensional statistics, the proposed methodology ensures that their solution provides mean-square stabilizing controllers with high probability even for low sample sizes. As sample size increases the closed-loop cost approaches that of the optimal controller produced wh…
This paper studies the sample complexity of the stochastic Linear Quadratic Regulator when applied to systems with multiplicative noise. We assume that the covariance of the noise is unknown and estimate it using the sample covariance, which results in suboptimal behaviour. The main contribution of this paper is then to bound the suboptimality of the methodology and prove that it decreases with 1/N, where N denotes the amount of samples. The methodology easily generalizes to the case where the mean is unknown and to the distributionally robust case studied in a previous work of the authors. Th…
A Data-Driven Algorithm for Model-Free Control Synthesis
Department of Electrical and Computer Engineering, Auburn University（电气与计算机工程系，阿伯杜大学）；Physics and Computational Sciences Division, Naval Air Warfare Center Weapons Division（物理与计算科学分部，海军航空武器中心武器分部）
This paper studies the robustness of policy iteration in the context of continuous-time infinite-horizon linear quadratic regulation (LQR) problem. It is shown that Kleinman's policy iteration algorithm is inherently robust to small disturbances and enjoys local input-to-state stability in the sense of Sontag. More precisely, whenever the disturbance-induced input term in each iteration is bounded and small, the solutions of the policy iteration algorithm are also bounded and enter a small neighborhood of the optimal solution of the LQR problem. Based on this result, an off-policy data-driven …
Harnessing Data from Clustered LQR Systems: Personalized and Collaborative Policy Optimization
Department of Electrical and Computer Engineering（电气与计算机工程系）；North Carolina State University（北卡罗来纳州立大学）；The Elmore Family School of Electrical and Computer Engineering（埃尔莫尔家族电气与计算机工程学院）；Purdue University（普渡大学）
With the continuous development of large-scale complex hybrid AC-DC grids, the fast adjustability of HVDC systems is required by the grid to provide frequency regulation services. This paper develops a fully data-driven linear quadratic regulator (LQR) for the HVDC to provide temporal frequency support. The main technical challenge is the complexity and the nonlinearity of multi-infeed hybrid AC-DC (MIDC) systems dynamics that make the LQR intractable. Based on Koopman operator (KO) theory, a Koopman eigenpairs construction method is developed to fit a global linear dynamic model of MIDC syste…
The goal of this paper is to develop data-driven control design and evaluation strategies based on linear matrix inequalities (LMIs) and dynamic programming. We consider deterministic discrete-time LTI systems, where the system model is unknown. We propose efficient data collection schemes from the state-input trajectories together with data-driven LMIs to design state-feedback controllers for stabilization and linear quadratic regulation (LQR) problem. In addition, we investigate theoretically guaranteed exploration schemes to acquire valid data from the trajectories under different scenarios…
In a recent paper we have shown that data collected from linear systems excited by persistently exciting inputs during low-complexity experiments, can be used to design state- and output-feedback controllers, including optimal Linear Quadratic Regulators (LQR), by solving linear matrix inequalities (LMI) and semidefinite programs. We have also shown how to stabilize in the first approximation unknown nonlinear systems using data. In contrast to the case of linear systems, however, in the case of nonlinear systems the conditions for learning a controller directly from data may not be fulfilled …
For various typical cases and situations where the formulation results in an optimal control problem, the Linear Quadratic Regulator (LQR) approach and its variants continue to be highly attractive. In certain scenarios, it can happen that some prescribed structural constraints on the gain matrix would arise. Consequently then, the Algebraic Riccati Equation (ARE) is no longer applicable in a straightforward way to obtain the optimal solution. This work presents a rather effective alternative optimization approach based on gradient projection. The utilized gradient is obtained through a data-d…
University of California, Santa Barbara（加州大学圣巴巴拉分校）
Soft robots promise improved safety and capability over rigid robots when deployed in complex, delicate, and dynamic environments. However, the infinite degrees of freedom and highly nonlinear dynamics of these systems severely complicate their modeling and control. As a step toward addressing this open challenge, we apply the data-driven, Hankel Dynamic Mode Decomposition (HDMD) with time delay observables to the model identification of a highly inertial, helical soft robotic arm with a high number of underactuated degrees of freedom. The resulting model is linear and hence amenable to contro…
Pacific Northwest National Laboratory (PNNL)（太平洋西北国家实验室）
This paper discusses learning a structured feedback control to obtain sufficient robustness to exogenous inputs for linear dynamic systems with unknown state matrix. The structural constraint on the controller is necessary for many cyber-physical systems, and our approach presents a design for any generic structure, paving the way for distributed learning control. The ideas from reinforcement learning (RL) in conjunction with control-theoretic sufficient stability and performance guarantees are used to develop the methodology. First, a model-based framework is formulated using dynamic programm…
Knowledge Integration in Differentiable Models: A Comparative Study of Data-Driven, Soft-Constrained, and Hard-Constrained Paradigms for Identification and Control of the Single Machine Infinite Bus System
Korea University（高丽大学）；Hyundai Motor Company（现代汽车公司）
Region of Attraction Estimation for Linear Quadratic Regulator, Linear and Robust Model Predictive Control on a Two-Wheeled Inverted Pendulum
Université Côte d‘Azur CNRS, I3S（法国蔚蓝海岸大学CNRS第三信息科学与系统研究所）；ETH Zürich IDSC（苏黎世联邦理工学院智能系统与控制系）；KTH Royal Institute of Technology Digital Futures（瑞典皇家理工学院数字未来中心）
