This paper studies asymptotic solvability of a linear quadratic (LQ) mean field social optimization problem with controlled diffusions and indefinite state and control weights. Starting with an $N$-agent model, we employ a rescaling approach to derive a low-dimensional Riccati ordinary differential equation (ODE) system, which characterizes a necessary and sufficient condition for asymptotic solvability. The decentralized control obtained from the mean field limit ensures a bounded optimality loss in minimizing the social cost having magnitude $O(N)$, which implies an optimality loss of $O(1/N)$ per agent. We further quantify the efficiency gain of the social optimum with respect to the solution of the mean field game.
We study the closed-loop solvability of a stochastic linear quadratic optimal control problem for systems governed by stochastic evolution equations. This solvability is established by means of solvability of the corresponding Riccati equation, which is implied by the uniform convexity of the quadratic cost functional. At last, conditions ensuring the uniform convexity of the cost functional are discussed.
This paper studies an asymptotic solvability problem for linear quadratic (LQ) mean field games with controlled diffusions and indefinite weights for the state and control in the costs. We employ a rescaling approach to derive a low dimensional Riccati ordinary differential equation (ODE) system, which characterizes a necessary and sufficient condition for asymptotic solvability. The rescaling technique is further used for performance estimates, establishing an $O(1/N)$-Nash equilibrium for the obtained decentralized strategies.
This paper investigates the so-called asymptotic solvability problem in linear quadratic (LQ) mean field games. The model has asymptotic solvability if for all sufficiently large population sizes, the corresponding game has a set of feedback Nash strategies subject to a mild regularity requirement. We provide a necessary and sufficient condition and show that in this case the solution converges to a mean field limit. This is accomplished by developing a re-scaling method to derive a low dimensional ordinary differential equation (ODE) system, where a non-symmetric Riccati ODE has a central rol…
This paper is concerned with the closed-loop solvability of one kind of linear-quadratic Stackelberg stochastic differential game, where the coefficients are deterministic. The notion of the closed-loop solvability is introduced, which require to be independent of the initial state. The follower's problem is solved first, and the closed-loop optimal strategy is characterized by a Riccati equation, together with an adapted solution to a linear backward stochastic differential equation. Then the necessary conditions of the existence of the leader's nonanticipating closed-loop optimal strategy is…
Recently it has been found that for a stochastic linear-quadratic optimal control problem (LQ problem, for short) in a finite horizon, open-loop solvability is strictly weaker than closed-loop solvability which is equivalent to the regular solvability of the corresponding Riccati equation. Therefore, when an LQ problem is merely open-loop solvable not closed-loop solvable, which is possible, the usual Riccati equation approach will fail to produce a state feedback representation of open-loop optimal controls. The objective of this paper is to introduce and investigate the notion of weak closed…
This paper is concerned with the stochastic linear-quadratic optimal control problem with Poisson jumps. The coefficients in the state equation and the weighting matrices in the cost functional are all deterministic but are allowed indefinite. The notion of closed-loop strategies is introduced, and the optimal closed-loop strategy is characterized by a Riccati integral-differential equation and a backward stochastic differential equation with Poisson jumps.
This paper is concerned with mean-field stochastic linear-quadratic (MF-SLQ, for short) optimal control problems with deterministic coefficients. The notion of weak closed-loop optimal strategy is introduced. It is shown that the open-loop solvability is equivalent to the existence of a weak closed-loop optimal strategy. Moreover, when open-loop optimal controls exist, there is at least one of them admitting a state feedback representation, which is the outcome of a weak closed-loop optimal strategy. Finally, an example is presented to illustrate the procedure for finding weak closed-loop opti…
An optimal control problem is studied for a linear mean-field stochastic differential equation with a quadratic cost functional. The coefficients and the weighting matrices in the cost functional are all assumed to be deterministic. Closed-loop strategies are introduced, which require to be independent of initial states; and such a nature makes it very useful and convenient in applications. In this paper, the existence of an optimal closed-loop strategy for the system (also called the closed-loop solvability of the problem) is characterized by the existence of a regular solution to the coupled…
This paper is concerned with two-person mean-field linear-quadratic non-zero sum stochastic differential games in an infinite horizon. Both open-loop and closed-loop Nash equilibria are introduced. Existence of an open-loop Nash equilibrium is characterized by the solvability of a system of mean-field forward-backward stochastic differential equations in an infinite horizon and the convexity of the cost functionals, and the closed-loop representation of an open-loop Nash equilibrium is given through the solution to a system of two coupled non-symmetric algebraic Riccati equations. The existenc…
Mean field game theory has been developed largely following two routes. One of them, called the direct approach, starts by solving a large-scale game and next derives a set of limiting equations as the population size tends to infinity. The second route is to apply mean field approximations and formalize a fixed point problem by analyzing the best response of a representative player. This paper addresses the connection and difference of the two approaches in a linear quadratic (LQ) setting. We first introduce an asymptotic solvability notion for the direct approach, which means for all suffici…
We provide conditions ensuring that the KKT-type conditions characterizes the global optimality for quadratically constrained (possibly nonconvex) quadratic programming QCQP problems in Hilbert spaces. The key property is the convexity of a image-type set related to the functions appearing in the formulation of the problem. The proof of the main result relies on a generalized version of the (Jakubovich) S-Lemma in Hilbert spaces. As an application, we consider the class of QCQP problems with a special form of the quadratic terms of the constraints.
In this paper, a leader-follower stochastic differential game is studied for a linear stochastic differential equation with a quadratic cost functional. The coefficients in the state equation and the weighting matrices in the cost functionals are all deterministic. Closed-loop strategies are introduced, which require to be independent of initial states; and such a nature makes it very useful and convenient in applications. The follower first solves a stochastic linear quadratic optimal control problem, and his optimal closed-loop strategy is characterized by a Riccati equation, together with a…
This paper is concerned with a stochastic linear quadratic (LQ, for short) optimal control problem. The notions of open-loop and closed-loop solvabilities are introduced. A simple example shows that these two solvabilities are different. Closed-loop solvability is established by means of solvability of the corresponding Riccati equation, which is implied by the uniform convexity of the quadratic cost functional. Conditions ensuring the convexity of the cost functional are discussed, including the issue that how negative the control weighting matrix-valued function R(s) can be. Finiteness of th…
This paper considers linear quadratic (LQ) mean field games with a major player and analyzes an asymptotic solvability problem. It starts with a large-scale system of coupled dynamic programming equations and applies a re-scaling technique introduced in Huang and Zhou (2018a, 2018b) to derive a set of Riccati equations in lower dimensions, the solvability of which determines the necessary and sufficient condition for asymptotic solvability. We next derive the mean field limit of the strategies and the value functions. Finally, we show that the two decentralized strategies can be interpreted as…
A linear quadratic optimal stochastic control problem with random coefficients and indefinite state/control weight costs is usually linked to an indefinite stochastic Riccati equation (SRE) which is a matrix-valued quadratic backward stochastic differential equation along with an algebraic constraint involving the unknown. Either the optimal control problem or the SRE is solvable only if the given data satisfy a certain structure condition that has yet to be precisely defined. In this paper, by introducing a notion of subsolution for the SRE, we derive several novel sufficient conditions for t…
In this paper, we investigate the open-loop and weak closed-loop solvabilities of stochastic linear quadratic (LQ, for short) optimal control problem of Markovian regime switching system. Interestingly, these two solvabilities are equivalent. We first provide an alternative characterization of the open-loop solvability of the LQ problem using the perturbation approach. Then, we study the weak closed-loop solvability of the LQ problem of Markovian regime switching system, and establish the equivalent relationship between open-loop and weak closed-loop solvabilities. Finally, we present an examp…
An optimal ergodic control problem (EC problem, for short) is investigated for a linear stochastic differential equation with quadratic cost functional. Constant nonhomogeneous terms, not all zero, appear in the state equation, which lead to the asymptotic limit of the state non-zero. Under the stabilizability condition, for any (admissible) closed-loop strategy, an invariant measure is proved to exist, which makes the ergodic cost functional well-defined and the EC problem well-formulated. Sufficient conditions, including those allowing the weighting matrices of cost functional to be indefini…
This paper is concerned with a linear quadratic stochastic two-person zero-sum differential game with constant coefficients in an infinite time horizon. Open-loop and closed-loop saddle points are introduced. The existence of closed-loop saddle points is characterized by the solvability of an algebraic Riccati equation with a certain stabilizing condition. A crucial result makes our approach work is the unique solvability of a class of linear backward stochastic differential equations in an infinite horizon.
This paper is concerned with a mean-field linear quadratic (LQ, for short) optimal control problem with deterministic coefficients. It is shown that convexity of the cost functional is necessary for the finiteness of the mean-field LQ problem, whereas uniform convexity of the cost functional is sufficient for the open-loop solvability of the problem. By considering a family of uniformly convex cost functionals, a characterization of the finiteness of the problem is derived and a minimizing sequence, whose convergence is equivalent to the open-loop solvability of the problem, is constructed. Th…
This paper is concerned with a new class of mean-field games which involve a finite number of agents. Necessary and sufficient conditions are obtained for the existence of the decentralized open-loop Nash equilibrium in terms of non-standard forward-backward stochastic differential equations (FBSDEs). By solving the FBSDEs, we design a set of decentralized strategies by virtue of two differential Riccati equations. Instead of the $\varepsilon$-Nash equilibrium in classical mean-field games, the set of decentralized strategies is shown to be a Nash equilibrium. For the infinite-horizon problem,…
This paper considers decentralized consensus optimization problems where nodes of a network have access to different summands of a global objective function. Nodes cooperate to minimize the global objective by exchanging information with neighbors only. A decentralized version of the alternating directions method of multipliers (DADMM) is a common method for solving this category of problems. DADMM exhibits linear convergence rate to the optimal objective but its implementation requires solving a convex optimization problem at each iteration. This can be computationally costly and may result i…
This paper considers an optimization problem that components of the objective function are available at different nodes of a network and nodes are allowed to only exchange information with their neighbors. The decentralized alternating method of multipliers (DADMM) is a well-established iterative method for solving this category of problems; however, implementation of DADMM requires solving an optimization subproblem at each iteration for each node. This procedure is often computationally costly for the nodes. We introduce a decentralized quadratic approximation of ADMM (DQM) that reduces comp…
This paper is concerned with a backward stochastic linear-quadratic (LQ, for short) optimal control problem with deterministic coefficients. The weighting matrices are allowed to be indefinite, and cross-product terms in the control and state processes are present in the cost functional. Based on a Hilbert space method, necessary and sufficient conditions are derived for the solvability of the problem, and a general approach for constructing optimal controls is developed. The crucial step in this construction is to establish the solvability of a Riccati-type equation, which is accomplished und…
This paper thoroughly investigates stochastic linear-quadratic optimal control problems with the Markovian regime switching system, where the coefficients of the state equation and the weighting matrices of the cost functional are random. We prove the solvability of the stochastic Riccati equation under the uniform convexity condition and obtain the closed-loop representation of the open-loop optimal control using the unique solvability of the corresponding stochastic Riccati equation. Moreover, by applying Itô's formula with jumps, we get a representation of the cost functional on a Hilbert s…
We provide an exhaustive treatment of Linear-Quadratic control problems for a class of stochastic Volterra equations of convolution type, whose kernels are Laplace transforms of certain signed matrix measures which are not necessarily finite. These equations are in general neither Markovian nor semimartingales, and include the fractional Brownian motion with Hurst index smaller than $1/2$ as a special case. We establish the correspondence of the initial problem with a possibly infinite dimensional Markovian one in a Banach space, which allows us to identify the Markovian controlled state varia…
An optimal control problem is considered for linear stochastic differential equations with quadratic cost functional. The coefficients of the state equation and the weights in the cost functional are bounded operators on the spaces of square integrable random variables. The main motivation of our study is linear quadratic optimal control problems for mean-field stochastic differential equations. Open-loop solvability of the problem is investigated, which is characterized as the solvability of a system of linear coupled forward-backward stochastic differential equations (FBSDE, for short) with …
This note concerns a class of matrix Riccati equations associated with stochastic linear-quadratic optimal control problems with indefinite state and control weighting costs. A novel sufficient condition of solvability of such equations is derived, based on a monotonicity property of a newly defined set. Such a set is used to describe a family of solvable equations.
In this paper, we investigate a class of time-inconsistent discrete-time stochastic linear-quadratic optimal control problems, whose time-consistent solutions consist of an open-loop equilibrium control and a linear feedback equilibrium strategy. The open-loop equilibrium control is defined for a given initial pair, while the linear feedback equilibrium strategy is defined for all the initial pairs. Maximum-principle-type necessary and sufficient conditions containing stationary and convexity are derived for the existence of these two time-consistent solutions, respectively. Furthermore, for t…
We consider the decentralized control of radial distribution systems with controllable photovoltaic inverters and energy storage resources. For such systems, we investigate the problem of designing fully decentralized controllers that minimize the expected cost of balancing demand, while guaranteeing the satisfaction of individual resource and distribution system voltage constraints. Employing a linear approximation of the branch flow model, we formulate this problem as the design of a decentralized disturbance-feedback controller that minimizes the expected value of a convex quadratic cost fu…
In this paper, we consider a linear quadratic stochastic two-person nonzero-sum differential game. Open-loop and closed-loop Nash equilibria are introduced. The existence of the former is characterized by the solvability of a system of forward-backward stochastic differential equations, and that of the latter is characterized by the solvability of a system of coupled symmetric Riccati differential equations. Sometimes, open-loop Nash equilibria admit a closed-loop representation, via the solution to a system of non-symmetric Riccati equations, which is different from the outcome of the closed-…
A linear quadratic (LQ) stochastic optimization problem with delay involving weakly-coupled large population is investigated in this paper. Different to classic mean field (MF) game, here agents cooperate with each other to minimize the so-called \emph{social} objective. With the aid of \emph{delayed person-by-person optimality} principle, one arrives at an auxiliary LQ delayed control problem by decentralized information. A decentralized strategy is obtained by feat of an MF type anticipated forward-backward stochastic differential delay equation (AFBSDDE) consistency condition. The discounti…
This paper is concerned with a linear quadratic optimal control problem of delayed backward stochastic differential equations. An explicit representation is derived for the optimal control, which is a linear feedback of the entire past history and the future state trajectory in a short period of time. This is one of the major distinctive features of the delayed backward stochastic linear quadratic optimal control problem. To obtain the optimal feedback, a new class of delayed Riccati equations is introduced and the unique solvability of their solutions are discussed in detail.
Mean field games with a major player were introduced in (Huang, 2010) within a linear-quadratic (LQ) modeling framework. Due to the rich structure of major-minor player models, the past ten years have seen significant research efforts for different solution notions and analytical techniques. For LQ models, we address the relation between three solution frameworks: the Nash certainty equivalence (NCE) approach in (Huang, 2010), master equations, and asymptotic solvability, which have been developed starting with different ideas. We establish their equivalence relationships.
This paper considers a linear-quadratic (LQ) mean field control problem involving a major player and a large number of minor players, where the dynamics and costs depend on random parameters. The objective is to optimize a social cost as a weighted sum of the individual costs under decentralized information. We apply the person-by-person optimality principle in team decision theory to the finite population model to construct two limiting variational problems whose solutions, subject to the requirement of consistent mean field approximations, yield a system of forward-backward stochastic differ…
As it is popular known, Riccati equation is the key basic tool for optimal control in the modern control theory. The solvability conditions of optimal control, stabilization conditions and controller design are all based on the Riccati equation. However, these results highly depends on a strictly assumption that the Riccati equation is regular. If the Riccati equation is irregular, the controller could not be derived from the equilibrium condition. This paper is concerned with the general stochastic linear quadratic (LQ) with irregular Riccati equation. Different from the classical control the…
University of Notre Dame（圣母大学）；University of Texas at Austin（德克萨斯大学奥斯汀分校）
We study the problem of control policy design for decentralized state-feedback linear quadratic control with a partially nested information structure, when the system model is unknown. We propose a model-based learning solution, which consists of two steps. First, we estimate the unknown system model from a single system trajectory of finite length, using least squares estimation. Next, based on the estimated system model, we design a control policy that satisfies the desired information structure. We show that the suboptimality gap between our control policy and the optimal decentralized cont…
