We study the problem of optimal state-feedback tracking control for unknown discrete-time deterministic systems with input constraints. To handle input constraints, state-of-art methods utilize a certain nonquadratic stage cost function, which is sometimes limiting real systems. Furthermore, it is well known that Policy Iteration (PI) and Value Iteration (VI), two widely used algorithms in data-driven control, offer complementary strengths and weaknesses. In this work, a two-step transformation is employed, which converts the constrained-input optimal tracking problem to an unconstrained augmented optimal regulation problem, and allows the consideration of general stage cost functions. Then, a novel multi-step VI algorithm based on Q-learning and linear programming is derived. The proposed algorithm improves the convergence speed of VI, avoids the requirement for an initial stabilizing control policy of PI, and computes a constrained optimal feedback controller without the knowledge of a system model and stage cost function. Simulation studies demonstrate the reliability and performance of the proposed approach.
Optimal tracking in switched systems with controlled subsystem and Discrete-time (DT) dynamics is investigated. A feedback control policy is generated such that a) the system tracks the desired reference signal, and b) the optimal switching time instants are sought. For finding the optimal solution, approximate dynamic programming is used. Simulation results are provided to illustrate the effectiveness of the solution.
The unknown sharp changes of vehicle acceleration rates, also called the unknown jerk dynamics, may significantly affect the driving performance of the leader vehicle in a platoon, resulting in more drastic car-following movements in platooning tracking control, which could cause safety and traffic capacity concerns. To address these issues, in this paper, we investigate cooperative platooning tracking control and intermittent optimization problems for connected automated vehicles (CAVs) with a nonlinear car-following model. We assume that the external inputs of the leader CAV contain unknown …
This paper proposes an adaptive dynamic programming-based adaptive-gain sliding mode control (ADP-ASMC) scheme for a fixed-wing unmanned aerial vehicle (UAV) with matched and unmatched disturbances. Starting from the dynamic of fixed-wing UAV, the control-oriented model composed of attitude subsystem and airspeed subsystem is established. According to the different issues in two subsystems, two novel adaptive-gain generalized super-twisting (AGST) algorithms are developed to eliminate the effects of disturbances in two subsystems and make the system trajectories tend to the designed integral s…
The isoline tracking of this work is concerned with the control design for a sensing robot to track a given isoline of an unknown 2-D scalar filed. To this end, we propose a coordinate-free controller with a simple PI-like form using only the concentration feedback for a Dubins robot, which is particularly useful in GPS-denied environments. The key idea lies in the novel design of a sliding surface based error term in the standard PI controller. Interestingly, we also prove that the tracking error can be reduced by increasing the proportion gain, and is eliminated for circular fields with a no…
We investigate a linear quadratic Gaussian (LQG) tracking problem with safety and reachability constraints in the presence of an adversary who mounts an FDI attack on an unknown set of sensors. For each possible set of compromised sensors, we maintain a state estimator disregarding the sensors in that set, and calculate the optimal LQG control input at each time based on this estimate. We propose a control policy which constrains the control input to lie within a fixed distance of the optimal control input corresponding to each state estimate. The control input is obtained at each time step by…
This paper studies a leader-follower formation tracking problem where the leaders are moving at the same unknown bounded velocity. A distance-based control law is proposed for follower agents to maintain the desired distances in the formation and move at the leaders' velocity. The control law consists of a component to handle the uncertainty of the leaders' velocity and a component to achieve the desired distances in finite time. Numerical simulations are also provided to support the theoretical results.
The isoline tracking of this work is concerned with the control design for a sensing vehicle to track a desired isoline of an unknown scalar field. To this end, we propose a simple PI-like controller for a Dubins vehicle in the GPS-denied environments. Our key idea lies in the design of a novel sliding surface based error in the standard PI controller. For the circular field, we show that the P-like controller can globally regulate the vehicle to the desired isoline with the steady-state error that can be arbitrarily reduced by increasing the P gain, and is eliminated by the PI-like controller…
We consider the problem of discounted optimal state-feedback regulation for general unknown deterministic discrete-time systems. It is well known that open-loop instability of systems, non-quadratic cost functions and complex nonlinear dynamics, as well as the on-policy behavior of many reinforcement learning (RL) algorithms, make the design of model-free optimal adaptive controllers a challenging task. We depart from commonly used least-squares and neural network approximation methods in conventional model-free control theory, and propose a novel family of data-driven optimization algorithms …
The optimal control input for linear systems can be solved from algebraic Riccati equation (ARE), from which it remains questionable to get the form of the exact solution. In engineering, the acceptable numerical solutions of ARE can be found by iteration or optimization. Recently, the gradient descent based numerical solutions has been proven effective to approximate the optimal ones. This paper introduces this method to tracking problem for heterogeneous linear systems. Differently, the parameters in the dynamics of the linear systems are all assumed to be unknown, which is intractable since…
We study a trajectory tracking problem for a multi-rotor in the presence of modeling error and external disturbances. The desired trajectory is unknown and generated from a reference system with unknown or partially known dynamics. We assume that only position and orientation measurements for the multi-rotor and position measurements for the reference system can be accessed. We adopt an extended high-gain observer (EHGO) estimation framework to estimate the feed-forward term required for trajectory tracking, the multi-rotor states, as well as modeling error and external disturbances. We design…
Optimal tracking of continuous time nonlinear systems has been extensively studied in literature. However, in several applications, absence of knowledge about system dynamics poses a severe challenge to solving the optimal tracking problem. This has found growing attention among researchers recently, and integral reinforcement learning (IRL)-based method augmented with actor neural network (NN) have been deployed to this end. However, very few studies have been directed to model-free $H_{\infty}$ optimal tracking control that helps in attenuating the effect of disturbances on the system perfor…
This paper focuses on the joint multi-object tracking (MOT) and the estimate of detection probability with the \emph{Poisson multi-Bernoulli mixture} (PMBM) filter. In a majority of multi-object scenarios, the knowledge of detection probability is usually uncertain, which is often estimated offline from the training data. In such cases, online filtering is not allowed or believable, otherwise, significate parameter mismatch will result in biased estimates (state and cardinality of objects). Consequently, the ability of adaptively estimating the detection probability is essential in practice. I…
Multi-Step Optimal Tracking Control of Unknown Nonzero-Sum Games based on Least Squares and Linear Programming: An Application to a Fully-Automated, Dual-Hormone Artificial Pancreas
Karlsruhe Institute of Technology (KIT)（卡尔斯鲁厄理工学院）
In order to autonomously learn to control unknown systems optimally w.r.t. an objective function, Adaptive Dynamic Programming (ADP) is well-suited to adapt controllers based on experience from interaction with the system. In recent years, many researchers focused on the tracking case, where the aim is to follow a desired trajectory. So far, ADP tracking controllers assume that the reference trajectory follows time-invariant exo-system dynamics-an assumption that does not hold for many applications. In order to overcome this limitation, we propose a new Q-function which explicitly incorporates…
Action-Increment-Constrained Policy Optimization for Smooth Online Adaptive Critic Flight Control with Actuator Rate Constraints and Low Computational Complexity (extended version with Ultimate Uniform Boundedness analysis)
Section Control and Simulation, Faculty of Aerospace Engineering, Delft University of Technology（控制与仿真部门，航空航天工程学院，代尔夫特理工大学）
In Model Predictive Control (MPC) formulations of trajectory tracking problems, infeasible reference trajectories and a-priori unknown constraints can lead to cumbersome designs, aggressive tracking, and loss of recursive feasibility. This is the case, for example, in trajectory tracking applications for mobile systems in the presence of constraints which are not fully known a-priori. In this paper, we propose a new framework called Model Predictive Flexible trajectory Tracking Control (MPFTC), which relaxes the trajectory tracking requirement. Additionally, we accommodate recursive feasibilit…
We propose a tracking control law for the fully actuated rigid body system in the presence of any unknown constant disturbance by employing quaternions with the stable embedding technique and Lyapunov stability theory. The stable embedding technique extends the attitude dynamics from the set of unit quaternions to the set of quaternions, which is a Euclidean space, such that the set of unit quaternions is an invariant set of the extended dynamics. Such a stable extension of the system dynamics to a Euclidean space allows us to employ well studied Lyapunov techniques in Euclidean spaces such as…
We address the problem of maintaining resource availability in a networked multi-robot team performing distributed tracking of unknown number of targets in an environment of interest. Based on our model, robots are equipped with sensing and computational resources enabling them to cooperatively track a set of targets in an environment using a distributed Probability Hypothesis Density (PHD) filter. We use the trace of a robot's sensor measurement noise covariance matrix to quantify its sensing quality. While executing the tracking task, if a robot experiences sensor quality degradation, then r…
This paper proposes a real-time neural network (NN) stochastic filter-based controller on the Lie Group of the Special Orthogonal Group $SO(3)$ as a novel approach to the attitude tracking problem. The introduced solution consists of two parts: a filter and a controller. Firstly, an adaptive NN-based stochastic filter is proposed that estimates attitude components and dynamics using measurements supplied by onboard sensors directly. The filter design accounts for measurement uncertainties inherent to the attitude dynamics, namely unknown bias and noise corrupting angular velocity measurements.…
In this paper, we will consider a class of continuous-time stochastic control systems with both unknown nonlinear structure and unknown disturbances, and investigate the capability of the classical proportional-integral-derivative(PID) controller in tracking time-varying reference signals. First, under some suitable conditions on system nonlinear functions, reference signals, and unknown disturbances, we will show that PID controllers can be designed to globally stabilize such systems and ensure the boundedness of the tracking error. Analytic design formulae for PID gain matrices are also prov…
We present a distributed optimization algorithm for solving online personalized optimization problems over a network of computing and communicating nodes, each of which linked to a specific user. The local objective functions are assumed to have a composite structure and to consist of a known time-varying (engineering) part and an unknown (user-specific) part. Regarding the unknown part, it is assumed to have a known parametric (e.g., quadratic) structure a priori, whose parameters are to be learned along with the evolution of the algorithm. The algorithm is composed of two intertwined compone…
In order to obviate the requirement of drift dynamics in adaptive dynamic programming (ADP), integral reinforcement learning (IRL) has been proposed as an alternate formulation of Bellman equation.However control coupling dynamics is still needed to obtain closed form expression of optimal control effort. In addition to this, initial stabilizing controller and two sets of neural networks (NN) (known as Actor-Critic) are required to implement IRL scheme. In this paper, a stabilizing term in the critic update law is leveraged to avoid the requirement of an initial stabilizing controller in IRL f…
Geometric Fault-Tolerant Neural Network Tracking Control of Unknown Systems on Matrix Lie Groups
Department of Mechanical, Industrial, and Mechatronics Engineering, Toronto Metropolitan University（机械、工业与机电工程系，多伦多 metropolitan 大学）；Department of Electrical Engineering, Amirkabir University of Technology（电气工程系，阿米卡比尔技术大学）
University of Pennsylvania（宾夕法尼亚大学）；Instituto de Ciencias Matematicas (CSIC-UAM-UCM-UC3M)（数学科学研究所（西班牙国家研究委员会-马德里自治大学-马德里康普顿斯大学-卡洛斯三世大学））；Technical University of Munich（慕尼黑工业大学）
Underactuated vehicles have gained much attention in the recent years due to the increasing amount of aerial and underwater vehicles as well as nanosatellites. Trajectory tracking control of these vehicles is a substantial aspect for an increasing range of application domains. However, external disturbances and parts of the internal dynamics are often unknown or very time-consuming to model. To overcome this issue, we present a tracking control law for underactuated rigid-body dynamics using an online learning-based oracle for the prediction of the unknown dynamics. We show that Gaussian proce…
Dual control explicitly addresses the problem of trading off active exploration and exploitation in the optimal control of partially unknown systems. While the problem can be cast in the framework of stochastic dynamic programming, exact solutions are only tractable for discrete state and action spaces of very small dimension due to a series of nested minimization and expectation operations. We propose an approximate dual control method for systems with continuous state and input domain based on a rollout dynamic programming approach, splitting the control horizon into a dual and an exploitati…
Accuracy and stability are common requirements for Quadrotor trajectory tracking systems. Designing an accurate and stable tracking controller remains challenging, particularly in unknown and dynamic environments with complex aerodynamic disturbances. We propose a Quantile-approximation-based Distributional-reinforced Uncertainty Estimator (QuaDUE) to accurately identify the effects of aerodynamic disturbances, i.e., the uncertainties between the true and estimated Control Contraction Metrics (CCMs). Taking inspiration from contraction theory and integrating the QuaDUE for uncertainties, our n…
Inspired by classical sensitivity results for nonlinear optimization, we derive and discuss new quantitative bounds to characterize the solution map and dual variables of a parametrized nonlinear program. In particular, we derive explicit expressions for the local and global Lipschitz constants of the solution map of non-convex or convex optimization problems, respectively. Our results are geared towards the study of time-varying optimization problems which are commonplace in various applications of online optimization, including power systems, robotics, signal processing and more. In this con…
We present a method for contraction-based feedback motion planning of locally incrementally exponentially stabilizable systems with unknown dynamics that provides probabilistic safety and reachability guarantees. Given a dynamics dataset, our method learns a deep control-affine approximation of the dynamics. To find a trusted domain where this model can be used for planning, we obtain an estimate of the Lipschitz constant of the model error, which is valid with a given probability, in a region around the training data, providing a local, spatially-varying model error bound. We derive a traject…
Indian Institute of Science（印度科学学院）；WIRIN Lab（WIRIN实验室）
This paper presents a coupled, neural network-aided longitudinal cruise and lateral path-tracking controller for an autonomous vehicle with model uncertainties and experiencing unknown external disturbances. Using a feedback error learning mechanism, an inverse vehicle dynamics learning scheme utilizing an adaptive Radial Basis Function (RBF) neural network, referred to as the Extended Minimal Resource Allocating Network (EMRAN) is employed. EMRAN uses an extended Kalman filter for online learning and weight updates, and also incorporates a growing/pruning strategy for maintaining a compact ne…
