Adaptive Dynamic Programming for Model-free Tracking of Trajectories with Time-varying Parameters
Karlsruhe Institute of Technology (KIT)（卡尔斯鲁厄理工学院）
Florian Köpf, Simon Ramsteiner, Michael Flad, Sören Hohmann
备注：
This is a preprint submitted to the Int J Adapt Control Signal Process. The substantially revised version will be published in the Int J Adapt Control Signal Process (DOI: 10.1002/ACS.3106)
In order to autonomously learn to control unknown systems optimally w.r.t. an objective function, Adaptive Dynamic Programming (ADP) is well-suited to adapt controllers based on experience from interaction with the system. In recent years, many researchers focused on the tracking case, where the aim is to follow a desired trajectory. So far, ADP tracking controllers assume that the reference trajectory follows time-invariant exo-system dynamics-an assumption that does not hold for many applications. In order to overcome this limitation, we propose a new Q-function which explicitly incorporates a parametrized approximation of the reference trajectory. This allows to learn to track a general class of trajectories by means of ADP. Once our Q-function has been learned, the associated controller copes with time-varying reference trajectories without need of further training and independent of exo-system dynamics. After proposing our general model-free off-policy tracking method, we provide analysis of the important special case of linear quadratic tracking. We conclude our paper with an example which demonstrates that our new method successfully learns the optimal tracking controller and outperforms existing approaches in terms of tracking error and cost.
Optimal tracking in switched systems with controlled subsystem and Discrete-time (DT) dynamics is investigated. A feedback control policy is generated such that a) the system tracks the desired reference signal, and b) the optimal switching time instants are sought. For finding the optimal solution, approximate dynamic programming is used. Simulation results are provided to illustrate the effectiveness of the solution.
University of Liverpool（利物浦大学）；Universidad Antonio de Nebrija（安东尼奥·德·内布里哈大学）；Chalmers University of Technology（查尔姆斯理工大学）；National Australia Bank（澳大利亚国民银行）
We propose a solution of the multiple target tracking (MTT) problem based on sets of trajectories and the random finite set framework. A full Bayesian approach to MTT should characterise the distribution of the trajectories given the measurements, as it contains all information about the trajectories. We attain this by considering multi-object density functions in which objects are trajectories. For the standard tracking models, we also describe a conjugate family of multitrajectory density functions.
We present a new quadrotor geometric control scheme that is capable of tracking highly aggressive trajectories. Our geometric controller uses the logarithmic map of SO(3) to express rotational error in the Lie algebra, and we show that it is globally attractive without requiring a complicated hybrid switching scheme. We show the performance of our controller against highly aggressive trajectories in simulation experiments. Additionally, we present an adaptation of this controller that allows us to interface effectively with the angular rate controllers on an onboard flight control unit and sho…
We study the problem of optimal state-feedback tracking control for unknown discrete-time deterministic systems with input constraints. To handle input constraints, state-of-art methods utilize a certain nonquadratic stage cost function, which is sometimes limiting real systems. Furthermore, it is well known that Policy Iteration (PI) and Value Iteration (VI), two widely used algorithms in data-driven control, offer complementary strengths and weaknesses. In this work, a two-step transformation is employed, which converts the constrained-input optimal tracking problem to an unconstrained augme…
This paper proposes an adaptive dynamic programming-based adaptive-gain sliding mode control (ADP-ASMC) scheme for a fixed-wing unmanned aerial vehicle (UAV) with matched and unmatched disturbances. Starting from the dynamic of fixed-wing UAV, the control-oriented model composed of attitude subsystem and airspeed subsystem is established. According to the different issues in two subsystems, two novel adaptive-gain generalized super-twisting (AGST) algorithms are developed to eliminate the effects of disturbances in two subsystems and make the system trajectories tend to the designed integral s…
Action-Increment-Constrained Policy Optimization for Smooth Online Adaptive Critic Flight Control with Actuator Rate Constraints and Low Computational Complexity (extended version with Ultimate Uniform Boundedness analysis)
Section Control and Simulation, Faculty of Aerospace Engineering, Delft University of Technology（控制与仿真部门，航空航天工程学院，代尔夫特理工大学）
This paper proposes a Poisson multi-Bernoulli mixture (PMBM) filter on the space of sets of tree trajectories for multiple target tracking with spawning targets. A tree trajectory contains all trajectory information of a target and its descendants, which appear due to the spawning process. Each tree contains a set of branches, where each branch has trajectory information of a target or one of the descendants and its genealogy. For the standard dynamic and measurement models with multi-Bernoulli spawning, the posterior is a PMBM density, with each Bernoulli having information on a potential tre…
Multi-Step Optimal Tracking Control of Unknown Nonzero-Sum Games based on Least Squares and Linear Programming: An Application to a Fully-Automated, Dual-Hormone Artificial Pancreas
Most passivity based trajectory tracking algorithms for mechanical systems can only stabilise reference trajectories that have constant energy. This paper overcomes this limitation by deriving a single variable Hamiltonian model for the reference trajectory and solving along the constrained trajectory to obtain a reference potential. This potential is then used as the model to shape the energy of the true system such that its free solutions include the desired reference trajectory. The proposed trajectory tracking algorithm interconnects the reference and true systems through a virtual spring …
A class of data-driven control methods has recently emerged based on Willems' fundamental lemma. Such methods can ease the modelling burden in control design but can be sensitive to disturbances acting on the system under control. In this paper, we propose a restructuring of the problem to incorporate segmented prediction trajectories. The proposed segmentation leads to reduced tracking error for longer prediction horizons in the presence of unmeasured disturbance and noise when compared to an unsegmented formulation. The performance characteristics are illustrated in a set-point tracking case…
Delft University of Technology（代尔夫特理工大学）；University of Leuven (KU Leuven)（鲁汶大学）
This paper proposes a new robust trajectory tracking error-based control approach for unmanned ground vehicles. A trajectory tracking error-based model is used to design a linear model predictive controller and its control action is combined with feedforward and robust control actions. The experimental results show that the proposed control structure is capable to let a tractor-trailer system track both linear and curvilinear target trajectories with low tracking error.
The goal of this paper is to develop data-driven control design and evaluation strategies based on linear matrix inequalities (LMIs) and dynamic programming. We consider deterministic discrete-time LTI systems, where the system model is unknown. We propose efficient data collection schemes from the state-input trajectories together with data-driven LMIs to design state-feedback controllers for stabilization and linear quadratic regulation (LQR) problem. In addition, we investigate theoretically guaranteed exploration schemes to acquire valid data from the trajectories under different scenarios…
We present a safe-by-design approach to path planning and control for nonlinear systems. The planner uses a low fidelity model of the plant to compute reference trajectories by solving an MPC problem, while the plant being controlled utilizes a feedback control law that tracks those trajectories with an upper-bound on the tracking error. Our main goal is to allow for maximum permissiveness (that is, room for constraint feasibility) of the planner, while maintaining safety after accounting for the tracking error bound. We achieve this by parametrizing the state and input constraints imposed on …
