$\text{H}_{\infty}$ Tracking Control via Variable Gain Gradient Descent-Based Integral Reinforcement Learning for Unknown Continuous Time Nonlinear System
Optimal tracking of continuous time nonlinear systems has been extensively studied in literature. However, in several applications, absence of knowledge about system dynamics poses a severe challenge to solving the optimal tracking problem. This has found growing attention among researchers recently, and integral reinforcement learning (IRL)-based method augmented with actor neural network (NN) have been deployed to this end. However, very few studies have been directed to model-free $H_{\infty}$ optimal tracking control that helps in attenuating the effect of disturbances on the system performance without any prior knowledge about system dynamics. To this end a recursive least square-based parameter update was recently proposed. However, gradient descent-based parameter update scheme is more sensitive to real-time variation in plant dynamics. And experience replay (ER) technique has been shown to improve the convergence of NN weights by utilizing past observations iteratively. Motivated by these, this paper presents a novel parameter update law based on variable gain gradient descent and experience replay technique for tuning the weights of critic, actor and disturbance NNs.
In the last few years, Motor Current Signature Analysis (MCSA) has proven to be an effective method for electrical machines condition monitoring. Indeed, many mechanical and electrical faults manifest as side-band spectral components generated around the fundamental frequency component of the motor current. These components are called interharmonics and they are a major focus of fault detection using MCSA. However, the main drawback of this approach is that the interference of other more prevalent components can obstruct the effect of interharmonics in the spectrum and may therefore impede fau…
This paper investigates a novel finite-time gradient descent-based adaptive neural network finite-time control strategy for the attitude tracking of a 3-DOF lab helicopter platform subject to composite disturbances. First, the radial basis function neural network (RBFNN) is applied to estimate lumped disturbances, where the weights, centers and widths of the RBFNN are trained online via finite-time gradient descent algorithm. Then, a finite-time backstepping control scheme is constructed to fulfill the tracking control of the elevation and pitch angles. In addition, a hybrid finite-time differ…
In recent times, a variety of Reinforcement Learning (RL) algorithms have been proposed for optimal tracking problem of continuous time nonlinear systems with input constraints. Most of these algorithms are based on the notion of uniform ultimate boundedness (UUB) stability, in which normally higher learning rates are avoided in order to restrict oscillations in state error to smaller values. However, this comes at the cost of higher convergence time of critic neural network weights. This paper addresses that problem by proposing a novel tuning law containing a variable gain gradient descent f…
Integral reinforcement learning (IRL) was proposed in literature to obviate the requirement of drift dynamics in adaptive dynamic programming framework. Most of the online IRL schemes in literature require two sets of neural network (NNs), known as actor-critic NN and an initial stabilizing controller. Recently, for RL-based robust tracking this requirement of initial stabilizing controller and dual-approximator structure could be obviated by using a modified gradient descent-based update law containing a stabilizing term with critic-only structure. To the best of the authors' knowledge, there…
The increasing electricity demand and the need for clean and renewable energy resources to satisfy this demand in a cost-effective manner, imposes new challenges on researchers and developers to maximize the output of these renewable resources at all times. However, the increasing penetration of renewable energy into the grid imposes new challenges on the grid operators. All of these challenges and issues gave rise to the need of Maximum Power Point Tracker (MPPT) and Flexible Power Point Trackers (FPPT) in order to maximize the power extracted from Photovoltaic (PV) systems and meet the grid …
The problem of attitude tracking using rotation matrices is addressed using an approach which combines inverse optimality and $\mathcal{L}_{2}$ disturbance attenuation. Conditions are provided which solve the inverse optimal nonlinear $H_{\infty}$ control problem by minimizing a meaningful cost function. The approach guarantees that the energy gain from an exogenous disturbance to a specified error signal respects a given upper bound. For numerical simulations, a simple problem setup from literature is considered and results demonstrate competitive performance.
This paper investigates the problem of tracking solutions of stochastic optimization problems with time-varying costs that depend on random variables with decision-dependent distributions. In this context, we propose the use of an online stochastic gradient descent method to solve the optimization, and we provide explicit bounds in expectation and in high probability for the distance between the optimizers and the points generated by the algorithm. In particular, we show that when the gradient error due to sampling is modeled as a sub-Weibull random variable, then the tracking error is ultimat…
Observer-Assisted Relative-Velocity Compensation with LPV-$H_\infty$ Robust Correction for 3D Trajectory Tracking of Underactuated Non-Minimum-Phase AUVs under Ocean Currents
We study the role that a finite timescale separation parameter $τ$ has on gradient descent-ascent in two-player non-convex, non-concave zero-sum games where the learning rate of player 1 is denoted by $γ_1$ and the learning rate of player 2 is defined to be $γ_2=τγ_1$. Existing work analyzing the role of timescale separation in gradient descent-ascent has primarily focused on the edge cases of players sharing a learning rate ($τ=1$) and the maximizing player approximately converging between each update of the minimizing player ($τ\rightarrow \infty$). For the parameter choice of $τ=1$, it is k…
The optimal control input for linear systems can be solved from algebraic Riccati equation (ARE), from which it remains questionable to get the form of the exact solution. In engineering, the acceptable numerical solutions of ARE can be found by iteration or optimization. Recently, the gradient descent based numerical solutions has been proven effective to approximate the optimal ones. This paper introduces this method to tracking problem for heterogeneous linear systems. Differently, the parameters in the dynamics of the linear systems are all assumed to be unknown, which is intractable since…
We consider the problem of tracking an adversarial state sequence in a linear dynamical system subject to adversarial disturbances and loss functions, generalizing earlier settings in the literature. To this end, we develop three techniques, each of independent interest. First, we propose a comparator-adaptive algorithm for online linear optimization with movement cost. Without tuning, it nearly matches the performance of the optimally tuned gradient descent in hindsight. Next, considering a related problem called online learning with memory, we construct a novel strongly adaptive algorithm th…
An external magnetic field can be used to remotely control small-scaled robots, making them promising candidates for diverse biomedical and engineering applications. We showed that our magnetically actuated millirobot is highly agile and can perform a variety of locomotive tasks such as pivot walking and tumbling in a horizontal plane. Here, we focus on controlling the locomotion outcomes of this millirobot in the pivot walking mode. A mathematical model of the system is developed and the kinematic model is derived. The role of the sweep and tilt angles in the robot's motion is also investigat…
University of Ottawa（渥太华大学）；Bradley University（布拉德利大学）
Underactuated systems like sea vessels have degrees of motion that are insufficiently matched by a set of independent actuation forces. In addition, the underlying trajectory-tracking control problems grow in complexity in order to decide the optimal rudder and thrust control signals. This enforces several difficult-to-solve constraints that are associated with the error dynamical equations using classical optimal tracking and adaptive control approaches. An online machine learning mechanism based on integral reinforcement learning is proposed to find a solution for a class of nonlinear tracki…
Texas A&M University（德克萨斯农工大学）；University of Minnesota（明尼苏达大学）
We consider a distributed non-convex optimization where a network of agents aims at minimizing a global function over the Stiefel manifold. The global function is represented as a finite sum of smooth local functions, where each local function is associated with one agent and agents communicate with each other over an undirected connected graph. The problem is non-convex as local functions are possibly non-convex (but smooth) and the Steifel manifold is a non-convex set. We present a decentralized Riemannian stochastic gradient method (DRSGD) with the convergence rate of $\mathcal{O}(1/\sqrt{K…
