In this paper, we consider an infinite horizon Linear-Quadratic-Gaussian control problem with controlled and costly measurements. A control strategy and a measurement strategy are co-designed to optimize the trade-off among control performance, actuating costs, and measurement costs. We address the co-design and co-optimization problem by establishing a dynamic programming equation with controlled lookahead. By leveraging the dynamic programming equation, we fully characterize the optimal control strategy and the measurement strategy analytically. The optimal control is linear in the state estimate that depends on the measurement strategy. We prove that the optimal measurement strategy is independent of the measured state and is periodic. And the optimal period length is determined by the cost of measurements and system parameters. We demonstrate the potential application of the co-design and co-optimization problem in an optimal self-triggered control paradigm. Two examples are provided to show the effectiveness of the optimal measurement strategy in reducing the overhead of measurements while keeping the system performance.
This paper presents a novel approach to synthesize dual controllers for unknown linear time-invariant systems with the tasks of optimizing a quadratic cost while reducing the uncertainty. To this end, a synthesis problem is defined where the feedback law has to simultaneously gain knowledge of the system and robustly optimize the cost. By framing the problem in a finite horizon setting, the trade-offs arising when the tasks include both identification and control are formally captured in the optimization problem. Results show that efficient exploration strategies are achieved when the structur…
The behaviour of a stochastic dynamical system may be largely influenced by those low-probability, yet extreme events. To address such occurrences, this paper proposes an infinite-horizon risk-constrained Linear Quadratic Regulator (LQR) framework with time-average cost. In addition to the standard LQR objective, the average one-stage predictive variance of the state penalty is constrained to lie within a user-specified level. By leveraging the duality, its optimal solution is first shown to be stationary and affine in the state, i.e., $u(x,λ^*) = -K(λ^*)x + l(λ^*)$, where $λ^*$ is an optimal …
An optimal control law for networked control systems with a discrete-time linear time-invariant (LTI) system as plant and networks between sensor and controller as well as between controller and actuator is proposed. This controller is designed by solving an optimization problem that is a generalization of the optimization problem used to obtain the Linear Quadratic Regulator (LQR) for deterministic discrete-time LTI systems. The networks are represented by random delays and drop outs of transmitted data packets.
We present a robust synthesis algorithm for uncertain linear time-varying (LTV) systems on finite horizons. The uncertain system is described as an interconnection of a known LTV system and a perturbation. The input-output behavior of the perturbation is specified by time-domain Integral Quadratic Constraints (IQCs). The objective is to synthesize a controller to minimize the worst-case performance. This leads to a non-convex optimization. The proposed approach alternates between an LTV synthesis step and an IQC analysis step. Both induced $\mathcal{L}_2$ and terminal Euclidean norm penalties …
The inverse linear-quadratic optimal control problem is a system identification problem whose aim is to recover the quadratic cost function and hence the closed-loop system matrices based on observations of optimal trajectories. In this paper, the discrete-time, finite-horizon case is considered, where the agents are also assumed to be homogeneous and indistinguishable. The latter means that the agents all have the same dynamics and objective functions and the observations are in terms of "snap shots" of all agents at different time instants, but what is not known is "which agent moved where" …
The goal of Inverse Optimal Control (IOC) is to identify the underlying objective function based on observed optimal trajectories. It provides a powerful framework to model expert's behavior, and a data-driven way to design an objective function so that the induced optimal control is adapted to a contextual environment. In this paper, we design an IOC algorithm for linear-quadratic tracking problems with random time horizon, and prove the statistical consistency of the algorithm. More specifically, the proposed estimator is the solution to a convex optimization problem, which means that the es…
In this work, we consider the problem of regret minimization in adaptive minimum variance and linear quadratic control problems. Regret minimization has been extensively studied in the literature for both types of adaptive control problems. Most of these works give results of the optimal rate of the regret in the asymptotic regime. In the minimum variance case, the optimal asymptotic rate for the regret is $\log(T)$ which can be reached without any additional external excitation. On the contrary, for most adaptive linear quadratic problems, it is necessary to add an external excitation in orde…
To address the issue of inaccurate distributions in practical stochastic systems, a minimax linear-quadratic control method is proposed using the Wasserstein metric. Our method aims to construct a control policy that is robust against errors in an empirical distribution of underlying uncertainty, by adopting an adversary that selects the worst-case distribution. The opponent receives a Wasserstein penalty proportional to the amount of deviation from the empirical distribution. A closed-form expression of the finite-horizon optimal policy pair is derived using a Riccati equation. The result is …
This paper studies the finite-horizon linear quadratic regulation problem where the dynamics of the system are assumed to be unknown and the state is accessible. Information on the system is given by a finite set of input-state data, where the input injected in the system is persistently exciting of a sufficiently high order. Using data, the optimal control law is then obtained as the solution of a suitable semidefinite program. The effectiveness of the approach is illustrated via numerical examples.
We address the problem of scheduling water resources in a power system via approximate dynamic programming.To this goal, we model a finite horizon economic dispatch problemwith convex stage cost and affine dynamics, and consider aquadratic approximation of the value functions. Evaluating theachieved policy entails solving a quadratic program at each timestep, while value function fitting can be cast as a semidefiniteprogram. We test our proposed algorithm on a simplified versionof the Uruguayan power system, achieving a four percent costreduction with respect to the myopic policy
Ensuring nominal asymptotic stability of the Nonlinear Model Predictive Control controller is not trivial. Stabilizing ingredients such as terminal penalty term and terminal region are crucial in establishing the asymptotic stability. Current work presents alternate approaches namely arbitrary controller based approach and linear quadratic regulator based approach, which provide larger degrees of freedom for enlarging the terminal region as against conservative approaches from the literature. Efficacy of the proposed approaches is demonstrated using benchmark two state continuous stirrer tank …
In the past couple of decades, non-quadratic convex penalties have reshaped signal processing and machine learning; in robust control, however, general convex costs break the Riccati and storage function structure that make the design tractable. Practitioners thus default to approximations, heuristics or robust model predictive control that are solved online for short horizons. We close this gap by extending $H_\infty$ control of discrete-time linear systems to strictly convex penalties on state, input, and disturbance, recasting the objective with Bregman divergences that admit a completion-o…
We propose a distributed data-based predictive control scheme to stabilize a network system described by linear dynamics. Agents cooperate to predict the future system evolution without knowledge of the dynamics, relying instead on learning a data-based representation from a single sample trajectory. We employ this representation to reformulate the finite-horizon Linear Quadratic Regulator problem as a network optimization with separable objective functions and locally expressible constraints. We show that the controller resulting from approximately solving this problem using a distributed opt…
An analysis of infinite horizon linear quadratic Gaussian (LQG) Mean Field Games is given within the general framework of Graphon Mean Field Games (GMFG) on dense infinite graphs (or networks) introduced in Caines and Huang (2018). For a class of LQG-GMFGs, analytical expressions are derived for the infinite horizon Nash values at the nodes of the infinite graph. Furthermore, under specific conditions on the network and the initial population means, it is shown that the nodes with strict local maximal infinite network degree are also nodes with strict local minimal cost at equilibrium.
This paper studies self-sustained dynamic multiagent systems (MAS) for decentralized resource allocation operating at a competitive equilibrium over a finite horizon. The utility of resource consumption, along with the income from resource exchange, forms each agent's payoff which is aimed to be maximized. Each utility function is parameterized by individual preferences which can be designed by agents independently. By shaping these preferences and proposing a set of utility functions, we can guarantee that the optimal resource price at the competitive equilibrium always remains socially accep…
Linear quadratic regulation of polytopic time-inhomogeneous Markov jump linear systems (extended version)
IMT School for Advanced Studies Lucca（卢卡高等研究学院）；Center of Excellence DEWS and with the Department of Information Engineering, Computer Science and Mathematics of the University of L’Aquila（拉奎拉大学信息与计算机工程及数学卓越中心）；Department of Computer Science of the University of Oxford（牛津大学计算机科学系）
We present an algorithm, based on the Differential Dynamic Programming framework, to handle trajectory optimization problems in which the horizon is determined online rather than fixed a priori. This algorithm exhibits exact one-step convergence for linear, quadratic, time-invariant problems and is fast enough for real-time nonlinear model-predictive control. We show derivations for the nonlinear algorithm in the discrete-time case, and apply this algorithm to a variety of nonlinear problems. Finally, we show the efficacy of the optimal-horizon model-predictive control scheme compared to a sta…
