This paper is concerned with linear quadratic graphon field social control problem where the noises of individual agents are correlated. Compared with the well-studied mean field system, the graphon field system consists of a large number of agents coupled weakly via a weighted undirected graph where each node represents an individual agent. Another notable feature of this paper is that the dynamics of states of agents are driven by Brownian motions with a correlation matrix. The infinite dimensional approach is adopted to design the centralized and decentralized controls for our large population system. By graphon theory, we prove that the linear quadratic (LQ) social optimum control problem under the centralized information pattern is equivalent to an LQ optimal control problem concerned with a stochastic evolution equation, and the feedback-type optimal centralized control is obtained. Then, by designing an auxiliary infinite dimensional optimal control problem through agent number $N\rightarrow\infty$, a set of decentralized strategies are constructed, which are further shown to be asymptotically social optimal.
In this paper we study the linear quadratic regulation (LQR) problem for dynamical systems coupled over large-scale networks and obtain locally computable low-complexity solutions. The underlying large or even infinite networks are represented by graphons and the couplings appear in both the dynamics and the quadratic cost. The optimal solution is obtained first for graphon dynamical systems for the special case where the graphons are exactly characterized by finite spectral summands. The complexity of generating these control solutions involves solving d+1 scalar Riccati equations where d is …
This paper studies approximate solutions to large-scale linear quadratic stochastic games with homogeneous nodal dynamics parameters and heterogeneous network couplings within the graphon mean field game framework in [2]-[4]. A graphon time-varying dynamical system model is first formulated to study the finite and then limit problems of linear quadratic Gaussian graphon mean field games (LQG-GMFG). The Nash equilibrium of the limit problem is then characterized by two coupled graphon time-varying dynamical systems. Sufficient conditions are established for the existence of a unique solution to…
To achieve control objectives for extremely large-scale complex networks using standard methods is essentially intractable. In this work a theory of the approximate control of complex network systems is proposed and developed by the use of graphon theory and the theory of infinite dimensional systems. First, graphon dynamical system models are formulated in an appropriate infinite dimensional space in order to represent arbitrary-size networks of linear dynamical systems, and to define the convergence of sequences of network systems with limits in the space. Exact controllability and approxima…
We consider static finite-player network games and their continuum analogs, graphon games. Existence and uniqueness results are provided, as well as convergence of the finite-player network game optimal strategy profiles to their analogs for the graphon games. We also show that equilibrium strategy profiles of a graphon game provide approximate Nash equilibria for the finite-player games. Connections with mean field games and central planner optimization problems are discussed. Motivating applications are presented and explicit computations of their Nash equilibria and social optimal strategie…
Graphon control has been proposed and developed in [1]-[3] to approximately solve control problems for very large-scale networks (VLSNs) of linear dynamical systems based on graphon limits. This paper provides a solution method based on invariant subspace decompositions for a class of graphon linear quadratic regulation (LQR) problems where the local dynamics share homogeneous parameters but the graphon couplings may be heterogeneous among the coupled agents. Graphon couplings in this paper appear in states, controls and costs, and these couplings may be represented by different graphons. By e…
A Deep Learning Method for Optimal Investment Under Relative Performance Criteria Among Heterogeneous Agents
Shanghai Frontiers Science Center of Artificial Intelligence and Deep Learning（上海人工智能与深度学习前沿科学中心）；NYU-ECNU Institute of Mathematical Sciences（纽约大学-华东师范大学数学科学研究所）；NYU Shanghai（上海纽约大学）；Princeton University（普林斯顿大学）
Recent advances at the intersection of dense large graph limits and mean field games have begun to enable the scalable analysis of a broad class of dynamical sequential games with large numbers of agents. So far, results have been largely limited to graphon mean field systems with continuous-time diffusive or jump dynamics, typically without control and with little focus on computational methods. We propose a novel discrete-time formulation for graphon mean field games as the limit of non-linear dense graph Markov games with weak interaction. On the theoretical side, we give extensive and rigo…
This paper is concerned with a linear quadratic (LQ, for short) optimal control problem with fixed terminal states and integral quadratic constraints. A Riccati equation with infinite terminal value is introduced, which is uniquely solvable and whose solution can be approximated by the solution for a suitable unconstrained LQ problem with penalized terminal state. Using results from duality theory, the optimal control is explicitly derived by solving the Riccati equation together with an optimal parameter selection problem. It turns out that the optimal control is not only a feedback of the cu…
In this paper, we consider linear quadratic optimal control with mean-field type for discrete-time stochastic systems with state and control dependent noise. An optimal control problem is studied for a linear mean-field stochastic differential equation with a quadratic cost functional. The coefficients and the weighting matrices in the cost functional are all assumed to be deterministic.
We study the equivalence of several well-known sufficient optimality conditions for a general quadratically constrained quadratic program (QCQP). The conditions are classified in two categories. The first one is for determining an optimal solution and the second one is for finding an optimal value. The first category of conditions includes the existence of a saddle point of the Lagrangian function and the existence of a rank-1 optimal solution of the primal SDP relaxation of QCQP. The second category includes $η_p = ζ$, $η_d = ζ$, and $φ= ζ$, where $ζ$, $η_p$, $η_d$, and $φ$ denote the optimal…
In a recent paper, Bubeck, Lee, and Singh introduced a new first order method for minimizing smooth strongly convex functions. Their geometric descent algorithm, largely inspired by the ellipsoid method, enjoys the optimal linear rate of convergence. We show that the same iterate sequence is generated by a scheme that in each iteration computes an optimal average of quadratic lower-models of the function. Indeed, the minimum of the averaged quadratic approaches the true minimum at an optimal rate. This intuitive viewpoint reveals clear connections to the original fast-gradient methods and cutt…
This paper considers the stochastic linear quadratic optimal control problem in which the control domain is nonconvex. By the functional analysis and convex perturbation methods, we establish a novel maximum principle. The application of the proposed maximum principle is illustrated through a work-out example.
Stochastic Optimal Linear Quadratic Regulation Control of Discrete-time Systems with Delay and Quadratic Constraints
School of Control Science and Engineering, Shandong University（山东大学控制科学与工程学院）；College of Electrical Engineering and Automation, Shandong University of Science and Technology（山东科技大学电气工程学院）
This paper is concerned with a linear quadratic optimal control problem of delayed backward stochastic differential equations. An explicit representation is derived for the optimal control, which is a linear feedback of the entire past history and the future state trajectory in a short period of time. This is one of the major distinctive features of the delayed backward stochastic linear quadratic optimal control problem. To obtain the optimal feedback, a new class of delayed Riccati equations is introduced and the unique solvability of their solutions are discussed in detail.
We study the closed-loop solvability of a stochastic linear quadratic optimal control problem for systems governed by stochastic evolution equations. This solvability is established by means of solvability of the corresponding Riccati equation, which is implied by the uniform convexity of the quadratic cost functional. At last, conditions ensuring the uniform convexity of the cost functional are discussed.
This paper is concerned with a linear quadratic optimal control for a class of singular Volterra integral equations. Under proper convexity conditions, optimal control uniquely exists, and it could be characterized via Frechet derivative of the quadratic functional in a Hilbert space or via maximum principle type necessary conditions. However, these (equivalent) characterizations have a shortcoming that the current value of the optimal control depends on the future values of the optimal state. Practically, this is not feasible. The main purpose of this paper is to obtain a causal state feedbac…
In this paper, we compare the strength of the optimal perspective reformulation and Shor's SDP relaxation. We prove these two formulations are equivalent for quadratic optimization problems with indicator variables.
This paper addresses an open problem in the area of linear quadratic optimal control. We consider the regular, infinite-horizon, stability-modulo-a-subspace, indefinite linear quadratic problem under the assumption that the dynamics are stabilizable. Our result generalizes previous works dealing with the same problem in the case of controllable dynamics. We explicitly characterize the unique solution of the algebraic Riccati equation that gives the optimal cost and optimal feedback control, as well as necessary and sufficient conditions for the existence of optimal controls.
A new method for solving optimal tracking control of linear quadratic time-varying systems with multiple time delays in state and input variables and with combined constraints is presented in this paper. By using the relations of Chebyshev wavelets, we simulate the optimal tracking problem to a static optimization one. This alternative method is applied on different optimal tracking systems and simulation results demonstrate the effectiveness of the proposed method.
This paper first presents necessary and sufficient conditions for the solvability of discrete time, mean-field, stochastic linear-quadratic optimal control problems. Then, by introducing several sequences of bounded linear operators, the problem becomes an operator stochastic LQ problem, in which the optimal control is a linear state feedback. Furthermore, from the form of the optimal control, the problem changes to a matrix dynamic optimization problem. Solving this optimization problem, we obtain the optimal feedback gain and thus the optimal control. Finally, by completing the square, the o…
In this paper, we define and solve the Inverse Stochastic Optimal Control (ISOC) problem of the linear-quadratic Gaussian (LQG) and the linear-quadratic sensorimotor (LQS) control model. These Stochastic Optimal Control (SOC) models are state-of-the-art approaches describing human movements. The LQG ISOC problem consists of finding the unknown weighting matrices of the quadratic cost function and the covariance matrices of the additive Gaussian noise processes based on ground truth trajectories observed from the human in practice. The LQS ISOC problem aims at additionally finding the covarianc…
Long-Time Behaviors of Stochastic Linear-Quadratic Optimal Control Problems
University of Michigan（密歇根大学）；California State University San Marcos（圣马科斯加州州立大学）；Worcester Polytechnic Institute（伍斯特理工学院）；University of Central Florida（中佛罗里达大学）
This paper is concerned with the stochastic linear-quadratic optimal control problem with Poisson jumps. The coefficients in the state equation and the weighting matrices in the cost functional are all deterministic but are allowed indefinite. The notion of closed-loop strategies is introduced, and the optimal closed-loop strategy is characterized by a Riccati integral-differential equation and a backward stochastic differential equation with Poisson jumps.
This paper is concerned with linear quadratic optimal control problems for mean-field backward stochastic differential equations (MF-BSDEs, for short) with deterministic coefficients. The optimality system, which is a linear mean-field forward-backward stochastic differential equation with constraint, is obtained by a variational method. By decoupling the optimality system, two coupled Riccati equations and an MF-BSDE are derived. It turns out that the coupled two Riccati equations are uniquely solvable. Then a complete and explicit representation is obtained for the optimal control.
We consider a general linear control system and a general quadratic cost, where the state evolves continuously in time and the control is sampled, i.e., is piecewise constant over a subdivision of the time interval. This is the framework of a linear-quadratic optimal sampled-data control problem. As a first result, we prove that, as the sampling periods tend to zero, the optimal sampled-data controls converge pointwise to the optimal permanent control. Then, we extend the classical Riccati theory to the sampled-data control framework, by developing two different approaches: the first one uses …
This paper is concerned with mean-field stochastic linear-quadratic (MF-SLQ, for short) optimal control problems with deterministic coefficients. The notion of weak closed-loop optimal strategy is introduced. It is shown that the open-loop solvability is equivalent to the existence of a weak closed-loop optimal strategy. Moreover, when open-loop optimal controls exist, there is at least one of them admitting a state feedback representation, which is the outcome of a weak closed-loop optimal strategy. Finally, an example is presented to illustrate the procedure for finding weak closed-loop opti…
This paper is concerned with a discrete-time mean-field stochastic linear-quadratic optimal control problem arose from financial application. Through matrix dynamical optimization method, a group of linear feedback controls is investigated. The problem is then reformulated as an operator stochastic linear-quadratic optimal control problem by a sequence of bounded linear operators over Hilbert space, the optimal control with six algebraic Riccati difference equations is obtained by backward induction. The two above approaches are proved to be coincided by the classical method of completing the …
This paper studies a continuous-time stochastic linear-quadratic (SLQ) optimal control problem on infinite-horizon. A data-driven policy iteration algorithm is proposed to solve the SLQ problem. Without knowing three system coefficient matrices, this algorithm uses the collected data to iteratively approximate a solution of the corresponding stochastic algebraic Riccati equation (SARE). A simulation example is provided to illustrate the effectiveness and applicability of the algorithm.
An alternating direction method of multipliers (ADMM) solver is described for optimal resource allocation problems with separable convex quadratic costs and constraints and linear coupling constraints. We describe a parallel implementation of the solver on a graphics processing unit (GPU) using a bespoke quartic function minimizer. An application to robust optimal energy management in hybrid electric vehicles is described, and the results of numerical simulations comparing the computation times of the parallel GPU implementation with those of an equivalent serial implementation are presented.
In this paper, we consider the inverse optimal control problem for the discrete-time linear quadratic regulator, over finite-time horizons. Given observations of the optimal trajectories, and optimal control inputs, to a linear time-invariant system, the goal is to infer the parameters that define the quadratic cost function. The well-posedness of the inverse optimal control problem is first justified. In the noiseless case, when these observations are exact, we analyze the identifiability of the problem and provide sufficient conditions for uniqueness of the solution. In the noisy case, when …
This paper is concerned with an optimal control problem for a mean-field linear stochastic differential equation with a quadratic functional in the infinite time horizon. Under suitable conditions, including the stabilizability, the (strong) exponential, integral, and mean-square turnpike properties for the optimal pair are established. The keys are to correctly formulate the corresponding static optimization problem and find the equations determining the correction processes. These have revealed the main feature of the stochastic problems which are significantly different from the determinist…
In this paper, we study a linear-quadratic optimal control problem for mean-field stochastic differential equations driven by a Poisson random martingale measure and a multidimensional Brownian motion. Firstly, the existence and uniqueness of the optimal control is obtained by the classic convex variation principle. Secondly, by the duality method, the optimality system, also called the stochastic Hamilton system which turns out to be a linear fully coupled mean-field forward-backward stochastic differential equation with jumps, is derived to characterize the optimal control. Thirdly, applying…
This paper investigates the social optimality of linear quadratic mean field control systems with unmodeled dynamics. The objective of agents is to optimize the social cost, which is the sum of costs of all agents. By variational analysis and direct decoupling methods, the social optimal control problem is analyzed, and two equivalent auxiliary robust optimal control problems are obtained for a representative agent. By solving the auxiliary problem with consistent mean field approximations, a set of decentralized strategies is designed, and its asymptotic social optimality is further proved. N…
This paper is concerned with a linear-quadratic (LQ, for short) optimal control problem for backward stochastic differential equations (BSDEs, for short), where the coefficients of the backward control system and the weighting matrices in the cost functional are allowed to be random. By a variational method, the optimality system, which is a coupled linear forward-backward stochastic differential equation (FBSDE, for short), is derived, and by a Hilbert space method, the unique solvability of the optimality system is obtained. In order to construct the optimal control, a new stochastic Riccati…
