The Linear Quadratic Regulator (LQR) framework considers the problem of regulating a linear dynamical system perturbed by environmental noise. We compute the policy regret between three distinct control policies: i) the optimal online policy, whose linear structure is given by the Ricatti equations; ii) the optimal offline linear policy, which is the best linear state feedback policy given the noise sequence; and iii) the optimal offline policy, which selects the globally optimal control actions given the noise sequence. We fully characterize the optimal offline policy and show that it has a recursive form in terms of the optimal online policy and future disturbances. We also show that cost of the optimal offline linear policy converges to the cost of the optimal online policy as the time horizon grows large, and consequently the optimal offline linear policy incurs linear regret relative to the optimal offline policy, even in the optimistic setting where the noise is drawn i.i.d from a known distribution. Although we focus on the setting where the noise is stochastic, our results also imply new lower bounds on the policy regret achievable when the noise is chosen by an adaptive adversary.
In this paper we design suboptimal control laws for an unknown linear system on the basis of measured data. We focus on the suboptimal linear quadratic regulator problem and the suboptimal H2 control problem. For both problems, we establish conditions under which a given data set contains sufficient information for controller design. We follow up by providing a data-driven parameterization of all suboptimal controllers. We will illustrate our results by numerical simulations, which will reveal an interesting trade-off between the number of collected data samples and the achieved controller per…
In the present work a new controller called Particle Swarm Optimization based state feedback gain controller has been proposed for frequency regulation of a two area system and then its performance is compared with earlier designed controllers such as Linear Quadratic Regulator Proportional Integral controller and Integral controller. The performance comparison has been done for the power system network comprising of two thermal power plants which are tie line connected. For using the optimal control based method such as LQR PI controller and computationally intelligent method such as PSO base…
The continuous and discrete time Linear Quadratic Regulator (LQR) theory has been used in this paper for the design of optimal analog and discrete PID controllers respectively. The PID controller gains are formulated as the optimal state-feedback gains, corresponding to the standard quadratic cost function involving the state variables and the controller effort. A real coded Genetic Algorithm (GA) has been used next to optimally find out the weighting matrices, associated with the respective optimal state-feedback regulator design while minimizing another time domain integral performance index…
In this paper, a polynomial chaos based framework for designing controllers for discrete time linear systems with probabilistic parameters is presented. Conditions for exponential-mean-square stability for such systems are derived and algorithms for synthesizing optimal quadratically stabilizing controllers are proposed in a convex optimization formulation. The solution presented is demonstrated on the derived discrete-time models of a nonlinear F-16 aircraft model trimmed at a set of chosen points.
Multi-objective LQR with Optimum Weight Selection to Design FOPID Controllers for Delayed Fractional Order Processes
School of Electronics and Computer Science, University of Southampton（南安普顿大学电子与计算机科学学院）；Department of Power Engineering, Jadavpur University（贾达普大学动力工程系）；Department of Earth Science and Engineering, Imperial College London（帝国理工学院地球科学与工程系）；Reactor Control Division, Bhabha Atomic Research Centre（巴哈原子研究中心反应堆控制部）
A novel conformal mapping based Fractional Order (FO) methodology is developed in this paper for tuning existing classical (Integer Order) Proportional Integral Derivative (PID) controllers especially for sluggish and oscillatory second order systems. The conventional pole placement tuning via Linear Quadratic Regulator (LQR) method is extended for open loop oscillatory systems as well. The locations of the open loop zeros of a fractional order PID (FOPID or PIλDμ) controller have been approximated in this paper vis-à-vis a LQR tuned conventional integer order PID controller, to achieve equiva…
Linear Quadratic Regulator (LQR) design is one of the most classical optimal control problems, whose well-known solution is an input sequence expressed as a state-feedback. In this work, finite-horizon and discrete-time LQR is solved under stability constraints and uncertain system dynamics. The resulting feedback controller balances cost value and closed-loop stability. Robustness of the solution is modeled using the scenario approach, without requiring any probabilistic description of the uncertainty in the system matrices. The new methods are tested and compared on the Leslie growth model, …
This paper presents a simulation-based comparison between the two controllers, Proportional Integral Derivative (PID), a classical controller and Linear Quadratic Regulator (LQR), an optimal controller, for a linearized quadrotor model. To simplify an otherwise complicated dynamic model of a quadrotor, we derive a linear mathematical model using Newtonian and Euler's laws and applying basic principles of physics. This derivation gives the equations that govern the motion of a quadrotor, both concerning the body frame and the inertial frame. A state-space model is developed, which is then used …
The Australian National University（澳大利亚国立大学）；The University of Texas at Dallas（德克萨斯大学达拉斯分校）
In this paper, we propose and analyze a new method for online linear quadratic regulator (LQR) control with a priori unknown time-varying cost matrices. The cost matrices are revealed sequentially with the potential for future values to be previewed over a short window. Our novel method involves using the available cost matrices to predict the optimal trajectory, and a tracking controller to drive the system towards it. We adopted the notion of dynamic regret to measure the performance of this proposed online LQR control method, with our main result being that the (dynamic) regret of our metho…
This paper presents a data-driven solution to the discrete-time infinite horizon LQR problem. The state feedback gain is computed directly from a batch of input and state data collected from the plant. Simulation examples illustrate the convergence of the proposed solution to the optimal LQR gain as the number of Markov parameters tends to infinity. Experiments in an uninterruptible power supply are presented, which demonstrate the practical applicability of the design methodology.
The linear-quadratic regulator (LQR) is an efficient control method for linear and linearized systems. Typically, LQR is implemented in minimal coordinates (also called generalized or "joint" coordinates). However, other coordinates are possible and recent research suggests that there may be numerical and control-theoretic advantages when using higher-dimensional non-minimal state parameterizations for dynamical systems. One such parameterization is maximal coordinates, in which each link in a multi-body system is parameterized by its full six degrees of freedom and joints between links are mo…
In this paper, we study the dynamic regret of online linear quadratic regulator (LQR) control with time-varying cost functions and disturbances. We consider the case where a finite look-ahead window of cost functions and disturbances is available at each stage. The online control algorithm studied in this paper falls into the category of model predictive control (MPC) with a particular choice of terminal costs to ensure the exponential stability of MPC. It is proved that the regret of such an online algorithm decays exponentially fast with the length of predictions. The impact of inaccurate pr…
We study the sample complexity of approximate policy iteration (PI) for the Linear Quadratic Regulator (LQR), building on a recent line of work using LQR as a testbed to understand the limits of reinforcement learning (RL) algorithms on continuous control tasks. Our analysis quantifies the tension between policy improvement and policy evaluation, and suggests that policy evaluation is the dominant factor in terms of sample complexity. Specifically, we show that to obtain a controller that is within $\varepsilon$ of the optimal LQR controller, each step of policy evaluation requires at most $(n…
We consider the problem of nonstochastic control with a sequence of quadratic losses, i.e., LQR control. We provide an efficient online algorithm that achieves an optimal dynamic (policy) regret of $\tilde{O}(\text{max}\{n^{1/3} \mathcal{TV}(M_{1:n})^{2/3}, 1\})$, where $\mathcal{TV}(M_{1:n})$ is the total variation of any oracle sequence of Disturbance Action policies parameterized by $M_1,...,M_n$ -- chosen in hindsight to cater to unknown nonstationarity. The rate improves the best known rate of $\tilde{O}(\sqrt{n (\mathcal{TV}(M_{1:n})+1)} )$ for general convex losses and we prove that it …
We present and solve a Linear Quadratic Regulator (LQR) for the boundary control of the beam equation. We use the simple technique of completing the square to get an explicit solution. By decoupling the spatial frequencies we are able to reduce an infinite dimensional LQR to an infinte family of two two dimensional LQRs each of which can be solved explicitly.
Birla Institute of Technology and Science – Pilani（贝拉理工学院皮尔尼分校）
This paper aims to design an optimal stability controller for a point to point trajectory tracking 3 degree of freedom articulated manipulator. The DH convention is used to obtain the forward and inverse kinematics of the manipulator. The manipulator dynamics are formulated using the Lagrange Euler method to obtain a nonlinear system. The complicated nonlinear equations obtained are then linearized in order to implement the optimal LQR. The simulations are performed in MATLAB and Simulink and the optimal controllers performance is tested for various conditions and the results are presented. Th…
We consider an LQR optimal control problem with partially unknown dynamics. We propose a new model-based online algorithm to obtain an approximation of the dynamics $and$ the control at the same time during a single simulation.
For linear time-invariant (LTI) systems, the design of an optimal controller is a commonly encountered problem in many applications. Among all the optimization approaches available, the linear quadratic regulator (LQR) methodology certainly garners much attention and interest. As is well-known, standard numerical tools in linear algebra are readily available which enable the determination of the optimal static LQR feedback gain matrix when all the system state variables are measurable. However, in various certain scenarios where some of the system state variables are not measurable, and conseq…
The present paper develops an optimal linear quadratic boundary controller for $2\times2$ linear hyperbolic partial differential equations (PDEs) with actuation on only one end of the domain. First-order necessary conditions for optimality is derived via weak variations and an optimal controller in state-feedback form is presented. The linear quadratic regulator (LQR) controller is calculated from differential algebraic Riccati equations. Numerical examples are performed to show the use of the proposed method.
In this article, we study the repeated routing game problem on a parallel network with affine latency functions on each edge. We cast the game setup in a LQR control theoretic framework, leveraging the Rosenthal potential formulation. We use control techniques to analyze the convergence of the game dynamics with specific cases that lend themselves to optimal control. We design proper dynamics parameters so that the conservation of flow is guaranteed. We provide an algorithmic solution for the general optimal control setup using a multiparametric quadratic programming approach (explicit MPC). F…
California Institute of Technology（加州理工学院）；Purdue University（普渡大学）
Thompson Sampling (TS) is an efficient method for decision-making under uncertainty, where an action is sampled from a carefully prescribed distribution which is updated based on the observed data. In this work, we study the problem of adaptive control of stabilizable linear-quadratic regulators (LQRs) using TS, where the system dynamics are unknown. Previous works have established that $\tilde O(\sqrt{T})$ frequentist regret is optimal for the adaptive control of LQRs. However, the existing methods either work only in restrictive settings, require a priori known stabilizing controllers, or ut…
Online optimization has recently opened avenues to study optimal control for time-varying cost functions that are unknown in advance. Inspired by this line of research, we study the distributed online linear quadratic regulator (LQR) problem for linear time-invariant (LTI) systems with unknown dynamics. Consider a multi-agent network where each agent is modeled as a LTI system. The network has a global time-varying quadratic cost, which may evolve adversarially and is only partially observed by each agent sequentially. The goal of the network is to collectively (i) estimate the unknown dynamic…
We propose a new risk-constrained reformulation of the standard Linear Quadratic Regulator (LQR) problem. Our framework is motivated by the fact that the classical (risk-neutral) LQR controller, although optimal in expectation, might be ineffective under relatively infrequent, yet statistically significant (risky) events. To effectively trade between average and extreme event performance, we introduce a new risk constraint, which explicitly restricts the total expected predictive variance of the state penalty by a user-prescribed level. We show that, under rather minimal conditions on the proc…
A Data-Driven Algorithm for Model-Free Control Synthesis
Department of Electrical and Computer Engineering, Auburn University（电气与计算机工程系，阿伯杜大学）；Physics and Computational Sciences Division, Naval Air Warfare Center Weapons Division（物理与计算科学分部，海军航空武器中心武器分部）
