A weighted summation of Integral of Time Multiplied Absolute Error (ITAE) and Integral of Squared Controller Output (ISCO) minimization based time domain optimal tuning of fractional-order (FO) PID or PIλDμ controller is proposed in this paper with a Linear Quadratic Regulator (LQR) based technique that minimizes the change in trajectories of the state variables and the control signal. A class of fractional order systems having single non-integer order element which show highly sluggish and oscillatory open loop responses have been tuned with an LQR based FOPID controller. The proposed controller design methodology is compared with the existing time domain optimal tuning techniques with respect to change in the trajectory of state variables, tracking performance for change in set-point, magnitude of control signal and also the capability of load disturbance suppression. A real coded genetic algorithm (GA) has been used for the optimal choice of weighting matrices while designing the quadratic regulator by minimizing the time domain integral performance index. Credible simulation studies have been presented to justify the proposition.
Multi-objective LQR with Optimum Weight Selection to Design FOPID Controllers for Delayed Fractional Order Processes
School of Electronics and Computer Science, University of Southampton（南安普顿大学电子与计算机科学学院）；Department of Power Engineering, Jadavpur University（贾达普大学动力工程系）；Department of Earth Science and Engineering, Imperial College London（帝国理工学院地球科学与工程系）；Reactor Control Division, Bhabha Atomic Research Centre（巴哈原子研究中心反应堆控制部）
In this paper we design suboptimal control laws for an unknown linear system on the basis of measured data. We focus on the suboptimal linear quadratic regulator problem and the suboptimal H2 control problem. For both problems, we establish conditions under which a given data set contains sufficient information for controller design. We follow up by providing a data-driven parameterization of all suboptimal controllers. We will illustrate our results by numerical simulations, which will reveal an interesting trade-off between the number of collected data samples and the achieved controller per…
In the present work a new controller called Particle Swarm Optimization based state feedback gain controller has been proposed for frequency regulation of a two area system and then its performance is compared with earlier designed controllers such as Linear Quadratic Regulator Proportional Integral controller and Integral controller. The performance comparison has been done for the power system network comprising of two thermal power plants which are tie line connected. For using the optimal control based method such as LQR PI controller and computationally intelligent method such as PSO base…
The continuous and discrete time Linear Quadratic Regulator (LQR) theory has been used in this paper for the design of optimal analog and discrete PID controllers respectively. The PID controller gains are formulated as the optimal state-feedback gains, corresponding to the standard quadratic cost function involving the state variables and the controller effort. A real coded Genetic Algorithm (GA) has been used next to optimally find out the weighting matrices, associated with the respective optimal state-feedback regulator design while minimizing another time domain integral performance index…
This paper presents a simulation-based comparison between the two controllers, Proportional Integral Derivative (PID), a classical controller and Linear Quadratic Regulator (LQR), an optimal controller, for a linearized quadrotor model. To simplify an otherwise complicated dynamic model of a quadrotor, we derive a linear mathematical model using Newtonian and Euler's laws and applying basic principles of physics. This derivation gives the equations that govern the motion of a quadrotor, both concerning the body frame and the inertial frame. A state-space model is developed, which is then used …
In this paper, a polynomial chaos based framework for designing controllers for discrete time linear systems with probabilistic parameters is presented. Conditions for exponential-mean-square stability for such systems are derived and algorithms for synthesizing optimal quadratically stabilizing controllers are proposed in a convex optimization formulation. The solution presented is demonstrated on the derived discrete-time models of a nonlinear F-16 aircraft model trimmed at a set of chosen points.
For linear time-invariant (LTI) systems, the design of an optimal controller is a commonly encountered problem in many applications. Among all the optimization approaches available, the linear quadratic regulator (LQR) methodology certainly garners much attention and interest. As is well-known, standard numerical tools in linear algebra are readily available which enable the determination of the optimal static LQR feedback gain matrix when all the system state variables are measurable. However, in various certain scenarios where some of the system state variables are not measurable, and conseq…
The Linear Quadratic Regulator (LQR) framework considers the problem of regulating a linear dynamical system perturbed by environmental noise. We compute the policy regret between three distinct control policies: i) the optimal online policy, whose linear structure is given by the Ricatti equations; ii) the optimal offline linear policy, which is the best linear state feedback policy given the noise sequence; and iii) the optimal offline policy, which selects the globally optimal control actions given the noise sequence. We fully characterize the optimal offline policy and show that it has a r…
We propose a new risk-constrained reformulation of the standard Linear Quadratic Regulator (LQR) problem. Our framework is motivated by the fact that the classical (risk-neutral) LQR controller, although optimal in expectation, might be ineffective under relatively infrequent, yet statistically significant (risky) events. To effectively trade between average and extreme event performance, we introduce a new risk constraint, which explicitly restricts the total expected predictive variance of the state penalty by a user-prescribed level. We show that, under rather minimal conditions on the proc…
A novel conformal mapping based Fractional Order (FO) methodology is developed in this paper for tuning existing classical (Integer Order) Proportional Integral Derivative (PID) controllers especially for sluggish and oscillatory second order systems. The conventional pole placement tuning via Linear Quadratic Regulator (LQR) method is extended for open loop oscillatory systems as well. The locations of the open loop zeros of a fractional order PID (FOPID or PIλDμ) controller have been approximated in this paper vis-à-vis a LQR tuned conventional integer order PID controller, to achieve equiva…
Stochastic optimal control usually requires an explicit dynamical model with probability distributions, which are difficult to obtain in practice. In this work, we consider the linear quadratic regulator (LQR) problem of unknown linear systems and adopt a Wasserstein penalty to address the distribution uncertainty of additive stochastic disturbances. By constructing an equivalent deterministic game of the penalized LQR problem, we propose a Q-learning method with convergence guarantees to learn an optimal minimax controller.
Stability is one of the most fundamental requirements for systems synthesis. In this paper, we address the stabilization problem for unknown linear systems via policy gradient (PG) methods. We leverage a key feature of PG for Linear Quadratic Regulator (LQR), i.e., it drives the policy away from the boundary of the unstabilizing region along the descent direction, provided with an initial policy with finite cost. To this end, we discount the LQR cost with a factor, by adaptively increasing which gradient leads the policy to the stabilizing set while maintaining a finite cost. Based on the Lyap…
Linear Quadratic Regulator (LQR) design is one of the most classical optimal control problems, whose well-known solution is an input sequence expressed as a state-feedback. In this work, finite-horizon and discrete-time LQR is solved under stability constraints and uncertain system dynamics. The resulting feedback controller balances cost value and closed-loop stability. Robustness of the solution is modeled using the scenario approach, without requiring any probabilistic description of the uncertainty in the system matrices. The new methods are tested and compared on the Leslie growth model, …
A Data-Driven Algorithm for Model-Free Control Synthesis
Department of Electrical and Computer Engineering, Auburn University（电气与计算机工程系，阿伯杜大学）；Physics and Computational Sciences Division, Naval Air Warfare Center Weapons Division（物理与计算科学分部，海军航空武器中心武器分部）
The linear-quadratic regulator (LQR) is an efficient control method for linear and linearized systems. Typically, LQR is implemented in minimal coordinates (also called generalized or "joint" coordinates). However, other coordinates are possible and recent research suggests that there may be numerical and control-theoretic advantages when using higher-dimensional non-minimal state parameterizations for dynamical systems. One such parameterization is maximal coordinates, in which each link in a multi-body system is parameterized by its full six degrees of freedom and joints between links are mo…
Designing the optimal linear quadratic regulator (LQR) for a large-scale multi-agent system (MAS) is time-consuming since it involves solving a large-size matrix Riccati equation. The situation is further exasperated when the design needs to be done in a model-free way using schemes such as reinforcement learning (RL). To reduce this computational complexity, we decompose the large-scale LQR design problem into multiple smaller-size LQR design problems. We consider the objective function to be specified over an undirected graph, and cast the decomposition as a graph clustering problem. The gra…
Indian Institute of Technology Kanpur（印度理工学院坎普尔分校）
We propose a framework for synthesizing a feedback control policy that maximizes the region of attraction (ROA) of a closed-loop nonlinear dynamical system. Our synthesis technique relies on stochastic optimization, which involves computation of an objective function capturing the ROA for a feedback control law. We employ a machine learning technique based on deep neural network to estimate the ROA for a given feedback controller. Overall, our technique is capable of synthesizing a controller co-optimizing traditional control objectives like LQR cost together with ROA. We demonstrate the effic…
We investigate the impact of packet dropouts due to non-idealities in communication networks on the performance of optimally derived controllers and observers in a minimax sense. These packet dropouts are modeled by discrete constrained switching signals via directed graphs. We consider time optimal control and estimation, minimum energy and fuel optimization and LQR problems for systems subject to packet dropouts which turn out to be combinatorial; and compute algorithmically the worst case performance of these optimization problems. To reduce the computational burden in solving these combina…
Towards Continuous Time Finite Horizon LQR Control in SE(3)
German Research Center for Artificial Intelligence (DFKI GmbH)（德国人工智能研究中心（DFKI GmbH））；Johannes Kepler University（约翰内斯·开普勒大学）；University of Notre Dame（圣母大学）；University of Bremen（不来梅大学）
Recently, reinforcement learning (RL) is receiving more and more attentions due to its successful demonstrations outperforming human performance in certain challenging tasks. In our recent paper `primal-dual Q-learning framework for LQR design,' we study a new optimization formulation of the linear quadratic regulator (LQR) problem via the Lagrangian duality theories in order to lay theoretical foundations of potentially effective RL algorithms. The new optimization problem includes the Q-function parameters so that it can be directly used to develop Q-learning algorithms, known to be one of t…
Birla Institute of Technology and Science – Pilani（贝拉理工学院皮尔尼分校）
This paper aims to design an optimal stability controller for a point to point trajectory tracking 3 degree of freedom articulated manipulator. The DH convention is used to obtain the forward and inverse kinematics of the manipulator. The manipulator dynamics are formulated using the Lagrange Euler method to obtain a nonlinear system. The complicated nonlinear equations obtained are then linearized in order to implement the optimal LQR. The simulations are performed in MATLAB and Simulink and the optimal controllers performance is tested for various conditions and the results are presented. Th…
In this paper a new optimum tuning method of PI controllers in first-order time-delay systems, based on the deadbeat response to a step setpoint variation, is presented. The deadbeat performance, already studied for the plants without delay, consists of a fast achievement of the steady state with an overshoot included in a narrow band and with minimum rise and settling times. In the proposed method the rise and settling times are both replaced with the integral of the squared error and the constraint, which requires a controller output less than a preset value, is added. The transient behaviou…
The present paper develops an optimal linear quadratic boundary controller for $2\times2$ linear hyperbolic partial differential equations (PDEs) with actuation on only one end of the domain. First-order necessary conditions for optimality is derived via weak variations and an optimal controller in state-feedback form is presented. The linear quadratic regulator (LQR) controller is calculated from differential algebraic Riccati equations. Numerical examples are performed to show the use of the proposed method.
University of California, Santa Cruz（加州大学圣克鲁兹分校）；Naval Postgraduate School（海军研究生院）
Recent research has shown that supervised learning can be an effective tool for designing optimal feedback controllers for high-dimensional nonlinear dynamic systems. But the behavior of these neural network (NN) controllers is still not well understood. In this paper we use numerical simulations to demonstrate that typical test accuracy metrics do not effectively capture the ability of an NN controller to stabilize a system. In particular, some NNs with high test accuracy can fail to stabilize the dynamics. To address this we propose two NN architectures which locally approximate a linear qua…
We study the sample complexity of approximate policy iteration (PI) for the Linear Quadratic Regulator (LQR), building on a recent line of work using LQR as a testbed to understand the limits of reinforcement learning (RL) algorithms on continuous control tasks. Our analysis quantifies the tension between policy improvement and policy evaluation, and suggests that policy evaluation is the dominant factor in terms of sample complexity. Specifically, we show that to obtain a controller that is within $\varepsilon$ of the optimal LQR controller, each step of policy evaluation requires at most $(n…
