In this paper, we present a Stochastic Deep Neural Network-based Model Reference Adaptive Control. Building on our work "Deep Model Reference Adaptive Control", we extend the controller capability by using Bayesian deep neural networks (DNN) to represent uncertainties and model non-linearities. Stochastic Deep Model Reference Adaptive Control uses a Lyapunov-based method to adapt the output-layer weights of the DNN model in real-time, while a data-driven supervised learning algorithm is used to update the inner-layers parameters. This asynchronous network update ensures boundedness and guaranteed tracking performance with a learning-based real-time feedback controller. A Bayesian approach to DNN learning helped avoid over-fitting the data and provide confidence intervals over the predictions. The controller's stochastic nature also ensured "Induced Persistency of excitation," leading to convergence of the overall system signal.
Bowling Green State University（博林格林州立大学）；University of Ottawa（渥太华大学）；Machine Intelligence Research Labs（机器智能研究实验室）
Model-reference adaptive systems refer to a consortium of techniques that guide plants to track desired reference trajectories. Approaches based on theories like Lyapunov, sliding surfaces, and backstepping are typically employed to advise adaptive control strategies. The resulting solutions are often challenged by the complexity of the reference model and those of the derived control strategies. Additionally, the explicit dependence of the control strategies on the process dynamics and reference dynamical models may contribute in degrading their efficiency in the face of uncertain or unknown …
In this paper, we present a novel idea to improve the transient performance of the existing Simple Adaptive Control architecture, without requiring high adaptation gains. Improvement in performance is achieved by incorporating the closed loop reference model based on the output feedback to the Simple Adaptive Control architecture. In this proposed scheme, the reference model dynamics is driven by the desired command as well as the error signal between the plant output and the reference model output. It is shown that the modified control architecture improves the system performance without any …
University of Illinois, Urbana-Champaign（伊利诺伊大学厄巴纳-香槟分校）
We present a new neuroadaptive architecture: Deep Neural Network based Model Reference Adaptive Control (DMRAC). Our architecture utilizes the power of deep neural network representations for modeling significant nonlinearities while marrying it with the boundedness guarantees that characterize MRAC based controllers. We demonstrate through simulations and analysis that DMRAC can subsume previously studied learning based MRAC methods, such as concurrent learning and GP-MRAC. This makes DMRAC a highly powerful architecture for high-performance control of nonlinear systems with long-term learnin…
This paper presents a framework for the design and analysis of an $\mathcal{L}_1$ adaptive controller with a switching reference system. The use of a switching reference system allows the desired behavior to be scheduled across the operating envelope, which is often required in aerospace applications. The analysis uses a switched reference system that assumes perfect knowledge of uncertainties and uses a corresponding non-adaptive controller. Provided that this switched reference system is stable, it is shown that the closed-loop system with unknown parameters and disturbances and the $\mathca…
In this paper, a model reference adaptive control architecture is proposed for uncertain nonlinear systems to achieve prescribed performance guarantees. Specifically, a general nonlinear reference model system is considered that captures an ideal and safe system behavior. An adaptive control architecture is then proposed to suppress the effects of system uncertainties without any prior knowledge of their magnitude and rate upper bounds. More importantly, the proposed control architecture enforces the system state trajectories to evolve within a user-specified prescribed distance from the refer…
Model-free control based on the idea of Reinforcement Learning is a promising approach that has recently gained extensive attention. However, Reinforcement-Learning-based control methods solely focus on the regulation problem or learn to track a reference that is generated by a time-invariant exo-system. In the latter case, controllers are only able to track the time-invariant reference dynamics which they have been trained on and need to be re-trained each time the reference dynamics change. Consequently, these methods fail in a number of applications which obviously rely on a trajectory not …
University of Illinois at Urbana-Champaign（伊利诺伊大学厄巴纳-香槟分校）
In this paper, we present Asynchronous implementation of Deep Neural Network-based Model Reference Adaptive Control (DMRAC). We evaluate this new neuro-adaptive control architecture through flight tests on a small quadcopter. We demonstrate that a single DMRAC controller can handle significant nonlinearities due to severe system faults and deliberate wind disturbances while executing high-bandwidth attitude control. We also show that the architecture has long-term learning abilities across different flight regimes, and can generalize to fly different flight trajectories than those on which it …
This paper proposes an adaptive control allocation approach for uncertain over-actuated systems with actuator saturation. The proposed method does not require uncertainty estimation or a persistent excitation assumption. Using the element-wise non-symmetric projection algorithm, the adaptive parameters are restricted to satisfy certain optimality conditions leading to overall closed loop system stability. Furthermore, a sliding mode controller with a time-varying sliding surface, working in tandem with the adaptive control allocation, is proposed to guarantee the outer loop stability and refer…
This paper presents an adaptive reference governor (RG) framework for a linear system with matched nonlinear uncertainties that can depend on both time and states, subject to both state and input constraints. The proposed framework leverages an L1 adaptive controller (L1AC) that estimates and compensates for the uncertainties, and provides guaranteed transient performance, in terms of uniform bounds on the error between actual states and inputs and those of a nominal (i.e., uncertainty-free) system. The uniform performance bounds provided by the L1AC are used to tighten the pre-specified state…
Robust Fixed-Time Model Reference Adaptive Control
Indian Institute of Technology Delhi（印度德里理工学院）；NIT Agartala（阿加塔拉国立技术学院）；Inria, Univ. Lille, CNRS, UMR 9189–CRIStAL（法国国家信息与自动化研究所，里尔大学，法国国家科学研究中心，CRIStAL实验室）
In this paper, we investigate the model reference adaptive control approach for uncertain piecewise affine systems with performance guarantees. The proposed approach ensures the error metric, defined as the weighted Euclidean norm of the state tracking error, to be confined within a user-defined time-varying performance bound. We introduce an auxiliary performance function to construct a barrier Lyapunov function. This auxiliary performance signal is reset at each switching instant, which prevents the transgression of the barriers caused by the jumps of the error metric at switching instants. …
Satisfaction of state and input constraints is one of the most critical requirements in control engineering applications. In classical model reference adaptive control (MRAC) formulation, although the states and the input remain bounded, the bound is neither user-defined nor known a-priori. In this paper, an MRAC is developed for multivariable linear time-invariant (LTI) plant with user-defined state and input constraints using a simple saturated control design coupled with a barrier Lyapunov function (BLF). Without any restrictive assumptions that may limit practical implementation, the propo…
In this paper we propose a new parameter estimator that ensures global exponential convergence of linear regression models requiring only the necessary assumption of identifiability of the regression equation,which we show is equivalent to interval excitation of the regressor vector. Continuous and discrete-time versions of the estimators are given. An extension to--separable and monotonic--non-linear parameterizations is also given. The estimators are shown to be robust to additive measurement noise and--not necessarily slow--parameter variations. Moreover, a version of the continuous-time es…
In December 2017, Seagate unveiled the Multi Actuator Technology to double the data performance of the future generation hard disk drives (HDD). This technology will equip drives with two dual stage actuators (DSA) each comprising of a voice coil motor (VCM) actuator and a piezoelectric micro actuator (MA) operating on the same pivot point. Each DSA is responsible for controlling half of the drive's arms. As both the DSAs operate independently on the same pivot timber, the control forces and torques generated by one can affect the operation of the other and thereby worsening the performance dr…
Adaptive Neural Control for a Class of Stochastic Nonlinear Systems with Unknown Parameters, Unknown Nonlinear Functions and Stochastic Disturbances
Hunan University of Science and Technology（湖南科技大学）；Central South University（中南大学）；Huazhong University of Science and Technology（华中科技大学）；Guangxi Teachers Education University（广西师范学院）
It is typically proven in adaptive control that asymptotic stabilization and tracking holds, and that at best a bounded-noise bounded-state property is proven. Recently, it has been shown in both the pole-placement control and the $d$-step ahead control settings that if, as part of the adaptive controller, a parameter estimator based on the original projection algorithm is used and the parameter estimates are restricted to a convex set, then the closed-loop system experiences linear-like behavior: exponential stability, a bounded gain on the noise in every $p$-norm, and a convolution bound on …
This paper presents a novel adaptive feedforward controller design for reset control systems. The combination of feedforward and reset feedback control promises high performance as the feedforward guarantees reference tracking, while the non-linear feedback element rejects disturbances. To overcome inevitable model mismatches, the feedforward controller adapts to increase precision in reference tracking. Where linear existing adaptive feedforward controllers do not guarantee convergence in the presence of reset, this work presents a novel adaptive law based on converging and diverging regions …
Stanford University（斯坦福大学）；Georgia Institute of Technology（佐治亚理工学院）
In this work we propose the use of adaptive stochastic search as a building block for general, non-convex optimization operations within deep neural network architectures. Specifically, for an objective function located at some layer in the network and parameterized by some network parameters, we employ adaptive stochastic search to perform optimization over its output. This operation is differentiable and does not obstruct the passing of gradients during backpropagation, thus enabling us to incorporate it as a component in end-to-end learning. We study the proposed optimization module's prope…
We propose a certainty-equivalence scheme for adaptive control of scalar linear systems subject to additive, i.i.d. Gaussian disturbances and bounded control input constraints, without requiring prior knowledge of the bounds of the system parameters, nor the control direction. Assuming that the system is at-worst marginally stable, mean square boundedness of the closed-loop system states is proven. Lastly, numerical examples are presented to illustrate our results.
Cornell University（康奈尔大学）；University of Connecticut（康涅狄格大学）
It is well known that adding any skew symmetric matrix to the gradient of Langevin dynamics algorithm results in a non-reversible diffusion with improved convergence rate. This paper presents a gradient algorithm to adaptively optimize the choice of the skew symmetric matrix. The resulting algorithm involves a non-reversible diffusion algorithm cross coupled with a stochastic gradient algorithm that adapts the skew symmetric matrix. The algorithm uses the same data as the classical Langevin algorithm. A weak convergence proof is given for the optimality of the choice of the skew symmetric matr…
This paper studies the adaptive optimal stationary control of continuous-time linear stochastic systems with both additive and multiplicative noises, using reinforcement learning techniques. Based on policy iteration, a novel off-policy reinforcement learning algorithm, named optimistic least-squares-based policy iteration, is proposed which is able to find iteratively near-optimal policies of the adaptive optimal stationary control problem directly from input/state data without explicitly identifying any system matrices, starting from an initial admissible control policy. The solutions given …
Successful control of a rigid-body rotating in three dimensional space requires accurate estimation of its attitude. The attitude dynamics are highly nonlinear and are posed on the Special Orthogonal Group $SO(3)$. In addition, measurements supplied by low-cost sensing units pose a challenge for the estimation process. This paper proposes a novel stochastic nonlinear neural-adaptive-based filter on $SO(3)$ for the attitude estimation problem. The proposed filter produces good results given measurements extracted from low-cost sensing units (e.g., IMU or MARG sensor modules). The filter is guar…
Learn-to-Fly (L2F) is a new framework that aims to replace the traditional iterative development paradigm for aerial vehicles with a combination of real-time aerodynamic modeling, guidance, and learning control. To ensure safe learning of the vehicle dynamics on the fly, this paper presents an $\mathcal{L}_1$ adaptive control ($\mathcal{L}_1$AC) based scheme, which actively estimates and compensates for the discrepancy between the intermediately learned dynamics and the actual dynamics. First, to incorporate the periodic update of the learned model within the L2F framework, this paper extends …
In this paper, we propose a chance constrained stochastic model predictive control scheme for reference tracking of distributed linear time-invariant systems with additive stochastic uncertainty. The chance constraints are reformulated analytically based on mean-variance information, where we design suitable Probabilistic Reachable Sets for constraint tightening. Furthermore, the chance constraints are proven to be satisfied in closed-loop operation. The design of an invariant set for tracking complements the controller and ensures convergence to arbitrary admissible reference points, while a …
