In this work we address the problem of the online robust control of nonlinear dynamical systems perturbed by disturbance. We study the problem of attenuation of the total cost over a duration $T$ in response to the disturbances. We consider the setting where the cost function (at a particular time) is a general continuous function and adversarial, the disturbance is adversarial and bounded at any point of time. Our goal is to design a controller that can learn and adapt to achieve a certain level of attenuation. We analyse two cases (i) when the system is known and (ii) when the system is unknown. We measure the performance of the controller by the deviation of the controller's cost for a sequence of cost functions with respect to an attenuation $γ$, $R^p_t$. We propose an online controller and present guarantees for the metric $R^p_t$ when the maximum possible attenuation is given by $\overlineγ$, which is a system constant. We show that when the controller has preview of the cost functions and the disturbances for a short duration of time and the system is known $R^p_T(γ) = O(1)$ when $γ\geq γ_c$, where $γ_c = \mathcal{O}(\overlineγ)$. We then show that when the system is unknown the proposed controller with a preview of the cost functions and the disturbances for a short horizon achieves $R^p_T(γ) = \mathcal{O}(N) + \mathcal{O}(1) + \mathcal{O}((T-N)g(N))$, when $γ\geq γ_c$, where $g(N)$ is the accuracy of a given nonlinear estimator and $N$ is the duration of the initial estimation period. We also characterize the lower bound on the required prediction horizon for these guarantees to hold in terms of the system constants.
We study a variant of online optimization in which the learner receives $k$-round $\textit{delayed feedback}$ about hitting cost and there is a multi-step nonlinear switching cost, i.e., costs depend on multiple previous actions in a nonlinear manner. Our main result shows that a novel Iterative Regularized Online Balanced Descent (iROBD) algorithm has a constant, dimension-free competitive ratio that is $O(L^{2k})$, where $L$ is the Lipschitz constant of the switching cost. Additionally, we provide lower bounds that illustrate the Lipschitz condition is required and the dependencies on $k$ an…
Robust control is a core approach for controlling systems with performance guarantees that are robust to modeling error, and is widely used in real-world systems. However, current robust control approaches can only handle small system uncertainty, and thus require significant effort in system identification prior to controller design. We present an online approach that robustly controls a nonlinear system under large model uncertainty. Our approach is based on decomposing the problem into two sub-problems, "robust control design" (which assumes small model uncertainty) and "chasing consistent …
In the framework of Model Predictive Control (MPC), the control input is typically computed by solving optimization problems repeatedly online. For general nonlinear systems, the online optimization problems are non-convex and computationally expensive or even intractable. In this paper, we propose to circumvent this issue by computing a high-dimensional linear embedding of discrete-time nonlinear systems. The computation relies on an algebraic condition related to the immersibility property of nonlinear systems and can be implemented offline. With the high-dimensional linear model, we then de…
This paper proposes a novel online motion planning approach to robot navigation based on nonlinear model predictive control. Common approaches rely on pure Euclidean optimization parameters. In robot navigation, however, state spaces often include rotational components which span over non-Euclidean rotation groups. The proposed approach applies nonlinear increment and difference operators in the entire optimization scheme to explicitly consider these groups. Realizations include but are not limited to quadratic form and time-optimal objectives. A complex parking scenario for the kinematic bicy…
Forced oscillations may jeopardize the secure operation of power systems. To mitigate forced oscillations, locating the sources is critical. In this paper, leveraging on Sparse Identification of Nonlinear Dynamics (SINDy), an online purely data-driven method to locate the forced oscillation is developed. Validations in all simulated cases (in the WECC 179-bus system) and actual oscillation events (in ISO New England system) in the IEEE Task Force test cases library are carried out, which demonstrate that the proposed algorithm, requiring no model information, can accurately locate sources in m…
We present a novel online learning algorithm for a class of unknown and uncertain dynamical environments that are fully observable. First, we obtain a novel probabilistic characterization of systems whose mean behavior is known but which are subject to additive, unknown subGaussian disturbances. This characterization relies on recent concentration of measure results and is given in terms of ambiguity sets. Second, we extend the results to environments whose mean behavior is also unknown but described by a parameterized class of possible mean behaviors. Our algorithm adapts the ambiguity set dy…
We present a framework to design nonlinear robust output feedback model predictive control (MPC) schemes that ensure constraint satisfaction under noisy output measurements and disturbances. We provide novel estimation methods to bound the magnitude of the estimation error based on: stability properties of the observer; detectability; set-membership estimation; moving horizon estimation (MHE). Robust constraint satisfaction is guaranteed by suitably incorporating these online validated bounds on the estimation error in a homothetic tube based MPC formulation. In addition, we show how the perfo…
We study the online robust control problem for linear dynamical systems with disturbances and uncertainties in the cost functions, with limited preview of the future disturbances and the cost functions, $N$. Our goal is to find an online control policy that can minimize the disturbance gain, defined as the ratio of the cumulative cost and the cumulative energy in the disturbances over a period of time, in the face of the uncertainties, and characterize its achievable gain in terms of the system relevant parameters. Our goals contrast with prior online control works for the same problem, which …
We propose a novel change point detection approach for online learning control with full information feedback (state, disturbance, and cost feedback) for unknown time-varying dynamical systems. We show that our algorithm can achieve a sub-linear regret with respect to the class of Disturbance Action Control (DAC) policies, which are a widely studied class of policies for online control of dynamical systems, for any sub-linear number of changes and very general class of systems: (i) matched disturbance system with general convex cost functions, (ii) general system with linear cost functions. Sp…
In this paper, we provide an algorithm for online computation of Koopman operator in real-time using streaming data. In recent years, there has been an increased interest in data-driven analysis of dynamical systems, with operator theoretic techniques being the most popular. Existing algorithms, like Dynamic Mode Decomposition (DMD) and Extended Dynamic Mode Decomposition (EDMD), use the entire data set for computation of the Koopman operator. However, many real life applications like power system analysis, biological systems, building systems etc. requires the real-time computation and updati…
The optimal tracking problem is addressed in the robotics literature by using a variety of robust and adaptive control approaches. However, these schemes are associated with implementation limitations such as applicability in uncertain dynamical environments with complete or partial model-based control structures, complexity and integrity in discrete-time environments, and scalability in complex coupled dynamical systems. An online adaptive learning mechanism is developed to tackle the above limitations and provide a generalized solution platform for a class of tracking control problems. This …
University of Notre Dame（圣母大学）；University of Hawaii（夏威夷大学）
Federated learning (FL) literature typically assumes that each client has a fixed amount of data, which is unrealistic in many practical applications. Some recent works introduced a framework for online FL (Online-Fed) wherein clients perform model learning on streaming data and communicate the model to the server; however, they do not address the associated communication overhead. As a solution, this paper presents a partial-sharing-based online federated learning framework (PSO-Fed) that enables clients to update their local models using continuous streaming data and share only portions of t…
In this paper, we present a nonlinear robust model predictive control (MPC) framework for general (state and input dependent) disturbances. This approach uses an online constructed tube in order to tighten the nominal (state and input) constraints. To facilitate an efficient online implementation, the shape of the tube is based on an offline computed incremental Lyapunov function with a corresponding (nonlinear) incrementally stabilizing feedback. Crucially, the online optimization only implicitly includes these nonlinear functions in terms of scalar bounds, which enables an efficient implemen…
This paper proposes a novel approach to estimate the steady-state angle stability limit (SSASL) by using the nonlinear power system dynamic model in the modal space. Through two linear changes of coordinates and a simplification introduced by the steady-state condition, the nonlinear power system dynamic model is transformed into a number of single-machine-like power systems whose power-angle curves can be derived and used for estimating the SSASL. The proposed approach estimates the SSASL of angles at all machines and all buses without the need for manually specifying the scenario, i.e. setti…
This work investigates the problem of estimating the weight matrices of a stable time-invariant linear dynamical system from a single sequence of noisy measurements. We show that if the unknown weight matrices describing the system are in Brunovsky canonical form, we can efficiently estimate the ground truth unknown matrices of the system from a linear system of equations formulated based on the transfer function of the system, using both online and offline stochastic gradient descent (SGD) methods. Specifically, by deriving concrete complexity bounds, we show that SGD converges linearly in ex…
Koopman Spectrum Nonlinear Regulators and Efficient Online Learning
University of Washington（华盛顿大学）；Ehime University（爱媛大学）；RIKEN Center for Advanced Intelligence Project（理化学研究所高级智能项目中心）；Et Cetera Robotics；Keio University（庆应义塾大学）；Harvard University（哈佛大学）；Osaka University（大阪大学）
University of California Irvine（加州大学欧文分校）；Texas A&M University（德州农工大学）
In this paper, we consider the problem of finding a meta-learning online control algorithm that can learn across the tasks when faced with a sequence of $N$ (similar) control tasks. Each task involves controlling a linear dynamical system for a finite horizon of $T$ time steps. The cost function and system noise at each time step are adversarial and unknown to the controller before taking the control action. Meta-learning is a broad approach where the goal is to prescribe an online policy for any new unseen task exploiting the information from other tasks and the similarity between the tasks. …
Stability-Preserving Online Adaptation of Neural Closed-loop Maps
Institute of Mechanical Engineering, Ecole Polytechnique Fédérale de Lausanne (EPFL)（洛桑联邦理工学院机械工程学院）；Department of Engineering Science, University of Oxford（牛津大学工程科学系）
