In control applications there is often a compromise that needs to be made with regards to the complexity and performance of the controller and the computational resources that are available. For instance, the typical hardware platform in embedded control applications is a microcontroller with limited memory and processing power, and for battery powered applications the control system can account for a significant portion of the energy consumption. We propose a controller architecture in which the computational cost is explicitly optimized along with the control objective. This is achieved by a three-part architecture where a high-level, computationally expensive controller generates plans, which a computationally simpler controller executes by compensating for prediction errors, while a recomputation policy decides when the plan should be recomputed. In this paper, we employ model predictive control (MPC) as the high-level plan-generating controller, a linear state feedback controller as the simpler compensating controller, and reinforcement learning (RL) to learn the recomputation policy. Simulation results for two examples showcase the architecture's ability to improve upon the MPC approach and find reasonable compromises weighing the performance on the control objective and the computational resources expended.
In Multi-Agent Reinforcement Learning (MA-RL), independent cooperative learners must overcome a number of pathologies to learn optimal joint policies. Addressing one pathology often leaves approaches vulnerable towards others. For instance, hysteretic Q-learning addresses miscoordination while leaving agents vulnerable towards misleading stochastic rewards. Other methods, such as leniency, have proven more robust when dealing with multiple pathologies simultaneously. However, leniency has predominately been studied within the context of strategic form games (bimatrix games) and fully observabl…
Skip What You Can Predict: Predictive Repositioning for Policy Optimization for Efficient LLM Training
Bangladesh University of Engineering and Technology（孟加拉工程与技术大学）；University of Maryland, College Park（马里兰大学学院公园分校）；Illinois Institute of Technology（伊利诺伊理工学院）；University of Central Florida（佛罗里达中央大学）
We consider the problem of stabilization of a linear system, under state and control constraints, and subject to bounded disturbances and unknown parameters in the state matrix. First, using a simple least square solution and available noisy measurements, the set of admissible values for parameters is evaluated. Second, for the estimated set of parameter values and the corresponding linear interval model of the system, two interval predictors are recalled and an unconstrained stabilizing control is designed that uses the predicted intervals. Third, to guarantee the robust constraint satisfacti…
In this manuscript, decentralized robust interval type-2 fuzzy model predictive control for Takagi-Sugeno large-scale systems is studied. The mentioned large-scale system consists a number of interval type-2 (IT2) fuzzy Takagi-Sugeno (T-S) subsystems. An important matter and necessities that limit the practical application of model predictive control are the online computational cost and burden of the existence frameworks. For model predictive control of T-S fuzzy large-scale systems, the online computational burden is even worse and in some cases, they cannot be solved in an expected time. Es…
In this paper, the model predictive control is designed for an interval type-2 Takagi-Sugeno (T-S) system with unknown time-varying delay in state and input vectors. The time-varying delay is a weird phenomenon that is appeared in almost all systems. It can make many problems and instability while the system is working. In this paper, the time-varying delay is considered in both states and input vectors and is the sensible difference between the proposed method here and previous algorithms, besides, it is unknown but bounded. To solve the problem, the Razumikhin approach is applied to the prop…
In this paper, we investigate the problem of Model Predictive Control (MPC) of dynamic systems for high-level specifications described by Signal Temporal Logic (STL) formulae. Recent works show that MPC has the great potential in handling logical tasks in reactive environments. However, existing approaches suffer from the heavy computational burden, especially for tasks with large horizons. In this work, we propose a computationally more efficient MPC framework for STL tasks based on time interval decomposition. Specifically, we still use the standard shrink horizon MPC framework with Mixed In…
Otto-von-Guericke-University（奥托·冯·格里克大学）；Universidad de Sevilla（塞维利亚大学）
Model predictive control allows to provide high performance and safety guarantees in the form of constraint satisfaction. These properties, however, can be satisfied only if the underlying model, used for prediction, of the controlled process is sufficiently accurate. One way to address this challenge is by data-driven and machine learning approaches, such as Gaussian processes, that allow to refine the model online during operation. We present a combination of an output feedback model predictive control scheme and a Gaussian process-based prediction model that is capable of efficient online l…
Many chemical processes exhibit diverse timescale dynamics with a strong coupling between timescale sensitive variables. Model predictive control with a non-uniformly spaced optimisation horizon is an effective approach to multi-timescale control and offers opportunities for reduced computational complexity. In such an approach the fast, moderate and slow dynamics can be included in the optimisation problem by implementing smaller time intervals earlier in the prediction horizon and increasingly larger intervals towards the end of the prediction. In this paper, a reference-flexible condition i…
We propose a computationally efficient Learning Model Predictive Control (LMPC) scheme for constrained optimal control of a class of nonlinear systems where the state and input can be reconstructed using lifted outputs. For the considered class of systems, we show how to use historical trajectory data collected during iterative tasks to construct a convex value function approximation along with a convex safe set in a lifted space of virtual outputs. These constructions are iteratively updated with historical data and used to synthesize predictive control policies. We show that the proposed str…
Sampling-based Model Predictive Control Using Trust Regions
Intelligent Sensor-Actuator-Systems Laboratory (ISAS), Institute for Anthropomatics and Robotics, Karlsruhe Institute of Technology（智能传感器执行器系统实验室，人机学与机器人研究所，卡尔斯鲁厄理工学院）
Sampling-based Model Predictive Control (MPC) is a flexible control framework that can reason about non-smooth dynamics and cost functions. Recently, significant work has focused on the use of machine learning to improve the performance of MPC, often through learning or fine-tuning the dynamics or cost function. In contrast, we focus on learning to optimize more effectively. In other words, to improve the update rule within MPC. We show that this can be particularly useful in sampling-based MPC, where we often wish to minimize the number of samples for computational reasons. Unfortunately, the…
We consider the effect of using approximate system predictions in event-triggered control schemes. Such approximations may result from using numerical transcription methods for solving continuous-time optimal control problems. Mesh refinement can guarantee upper bounds on the error in the differential equations which model the system dynamics. With the accuracy guarantees of a mesh refinement scheme, we show that the proposed event-triggering scheme -- which compares the measured system with approximate state predictions -- can be used with a guaranteed strictly positive inter-update time. We …
This paper considers the control of uncertain systems that are operated under limited resource factors, such as battery life or hardware longevity. We consider here resource-aware self-triggered control techniques that schedule system operation non-uniformly in time in order to balance performance against resource consumption. When running in an uncertain environment, unknown disturbances may deteriorate system performance by acting adversarially against the planned event triggering schedule. In this work, we propose a resource-aware stochastic predictive control scheme to tackle this challeng…
In this paper we present a dual active-set solver for quadratic programming which has properties suitable for use in embedded model predictive control applications. In particular, the solver is efficient, can easily be warm-started, and is simple to code. Moreover, the exact worst-case computational complexity of the solver can be determined offline and, by using outer proximal-point iterations, ill-conditioned problems can be handled in a robust manner.
Michigan State University（密歇根州立大学）；Utah State University（犹他州立大学）
This paper presents a sample-efficient data-driven method to design model predictive control (MPC) for cable-actuated soft robotics using Bayesian optimization. Instead of modeling the complex dynamics of the soft robots, the proposed approach uses Bayesian optimization to search the best-guessed low-dimensional prediction model and its associated controller to minimize the objective function of closed-loop responses. The prediction model is updated by Bayesian optimization from the closed-loop input-output data in each iteration. A linear MPC is then designed based on the updated prediction m…
Robotic Systems Lab, ETH Zurich（苏黎世联邦理工学院机器人系统实验室）
The Sequential Linear Quadratic (SLQ) algorithm is a continuous-time variant of the well-known Differential Dynamic Programming (DDP) technique with a Gauss-Newton Hessian approximation. This family of methods has gained popularity in the robotics community due to its efficiency in solving complex trajectory optimization problems. However, one major drawback of DDP-based formulations is their inability to properly incorporate path constraints. In this paper, we address this issue by devising a constrained SLQ algorithm that handles a mixture of constraints with a previously implemented project…
In this paper, a control algorithm for guiding a two wheeled mobile robot with unknown inertia to a desired point and orientation using an Adaptive Model Predictive Control (AMPC) framework is presented. The two wheeled mobile robot is modeled as a knife edge or a skate with nonholonomic kinematic constraints and the dynamical equations are derived using the Lagrangian approach. The inputs at every time instant are obtained from Model Predictive Control (MPC) with a set of nominal parameters which are updated using a recursive least squares algorithm. The efficacy of the algorithm is demonstra…
