This paper studies a two-player zero-sum stochastic linear-quadratic (SLQ) differential game for controlled jump-diffusion systems with random coefficients, where the controls of both players are constrained to nonempty closed convex cones. Under a uniform convexity--concavity condition, we establish the existence and uniqueness of an open-loop saddle point and characterize it by a forward--backward stochastic differential equation with jumps (FBSDEJ) together with cone-type variational inequalities. Assuming the existence of positive bounded solutions to the associated system of indefinite extended stochastic Riccati equations with jumps (IESREJs), we derive a feedback-form representation of the unique open-loop saddle point by constructing predictable minimax selectors and combining the Meyer--Itô formula with jumps, and the FBSDEJ characterization. Finally, under additional structural conditions, we prove the existence of positive bounded solutions to the IESREJs by a double-truncation approximation and a multidimensional BSDEJ comparison theorem.
This paper is concerned with stochastic differential games (SDGs) defined through fully coupled forward-backward stochastic differential equations (FBSDEs) which are governed by Brownian motion and Poisson random measure. For SDGs, the upper and the lower value functions are defined by the controlled fully coupled FBSDEs with jumps. Using a new transformation introduced in [6], we prove that the upper and the lower value functions are deterministic. Then, after establishing the dynamic programming principle for the upper and the lower value functions of this SDGs, we prove that the upper and t…
We study the discrete-time approximation for solutions of quadratic forward back- ward stochastic differential equations (FBSDEs) driven by a Brownian motion and a jump process which could be dependent. Assuming that the generator has a quadratic growth w.r.t. the variable z and the terminal condition is bounded, we prove the convergence of the scheme when the number of time steps n goes to infinity. Our approach is based on the companion paper [15] and allows to get a convergence rate similar to that of schemes of Brownian FBSDEs.
This paper considers a two-person zero-sum continuous-time Markov pure jump game in Borel state and action spaces over a fixed finite horizon. The main assumption on the model is the existence of a drift function, which bounds the reward rate. Under some regularity conditions, we show that the game has a value, and both of the players have their optimal policies.
In this paper, we investigate infinite horizon jump-diffusion forward-backward stochastic differential equations under some monotonicity conditions. We establish an existence and uniqueness theorem, two stability results and a comparison theorem for solutions to such kind of equations. Then the theoretical results are applied to study a kind of infinite horizon backward stochastic linear-quadratic optimal control problems, and then differential game problems. The unique optimal controls for the control problems and the unique Nash equilibrium points for the game problems are obtained in closed…
A zero-sum differential game with controlled jump-diffusion driven state is considered, and studied using a combination of dynamic programming and viscosity solution techniques. We prove, under certain conditions, that the value of the game exists and is the unique viscosity solution of a fully nonlinear integro-partial differential equation. In addition, we formulate and prove a verification theorem for such games within the viscosity solution framework for nonlocal equations
In this paper we study zero-sum two-player stochastic differential games with jumps with the help of theory of Backward Stochastic Differential Equations (BSDEs). We generalize the results of Fleming and Souganidis [10] and those by Biswas [3] by considering a controlled stochastic system driven by a d-dimensional Brownian motion and a Poisson random measure and by associating nonlinear cost functionals defined by controlled BSDEs. Moreover, unlike the both papers cited above we allow the admissible control processes of both players to depend on all events occurring before the beginning of the…
We develop an approach for two player constraint zero-sum and nonzero-sum stochastic differential games, which are modeled by Markov regime-switching jump-diffusion processes. We provide the relations between a usual stochastic optimal control setting and a Lagrangian method. In this context, we prove corresponding theorems for two different type of constraints, which lead us to find real valued and stochastic Lagrange multipliers, respectively. Then, we illustrate our results for a nonzero-sum game problem with stochastic maximum principle technique. Our application is an example of cooperati…
We investigate mean-field games from the point of view of a large number of indistinguishable players which eventually converges to infinity. The players are weakly coupled via their empirical measure. The dynamics of the states of the individual players is governed by a non-autonomous pure jump type semi group in a Euclidean space, which is not necessarily smoothing. Investigations are conducted in the framework of non-linear Markov processes. We show that the individual optimal strategy results from a consistent coupling of an optimal control problem with a forward non-autonomous dynamics. I…
This paper is concerned with the stochastic linear-quadratic optimal control problem with Poisson jumps. The coefficients in the state equation and the weighting matrices in the cost functional are all deterministic but are allowed indefinite. The notion of closed-loop strategies is introduced, and the optimal closed-loop strategy is characterized by a Riccati integral-differential equation and a backward stochastic differential equation with Poisson jumps.
We consider a couple of integrodifferential PDEs arising from a stochastic Markovian control problem subjected to initial-terminal conditions. These equations correspond to the MFG system for a controlled jump-diffusion process. We prove that for a specific choice of the control function the expectation of the jump-diffusion process can be found explicitly. The study is an extension of similar results known for the pure diffusion process. As an example, we show how this can be applied to the problem of investors evaluating the trend of an asset when choosing an optimal portfolio.
In this paper, we study a class of linear-quadratic (LQ) mean-field games in which the individual control process is constrained in a closed convex subset $Γ$ of full space $\mathbb{R}^m$. The decentralized strategies and consistency condition are represented by a class of mean-field forward-backward stochastic differential equation (MF-FBSDE) with projection operators on $Γ$. The wellposedness of consistency condition system is obtained using the monotonicity condition method. The related $ε$-Nash equilibrium property is also verified.
This work develops an approximation procedure for a class of non-zero-sum stochastic differential investment and reinsurance games between two insurance companies. Both proportional reinsurance and excess-of loss reinsurance policies are considered. We develop numerical algorithms to obtain the Nash equilibrium by adopting the Markov chain approximation methodology and applying the dynamical programming principle for the nonlinear integro-differential Hamilton-Jacobi-Isaacs (HJI) equations. Furthermore, we establish the convergence of the approximation sequences and the approximation to the va…
In this paper, we investigate the solvability of matrix valued Backward stochastic Riccati equations with jumps (BSREJ), which is associated with a stochastic linear quadratic (SLQ) optimal control problem with random coefficients and driven by both Brownian motion and Poisson jumps. By dynamic programming principle, Doob-Meyer decomposition and inverse flow technique, the existence and uniqueness of the solution for the BSREJ is established. The difficulties addressed to this issue not only are brought from the high nonlinearity of the generator of the BSREJ like the case driven only by Brown…
We consider a multi-player stochastic differential game with linear McKean-Vlasov dynamics and quadratic cost functional depending on the variance and mean of the state and control actions of the players in open-loop form. Finite and infinite horizon problems with possibly some random coefficients as well as common noise are addressed. We propose a simple direct approach based on weak martingale optimality principle together with a fixed point argument in the space of controls for solving this game problem. The Nash equilibria are characterized in terms of systems of Riccati ordinary different…
This paper studies the n-player game and the mean field game under the CRRA relative performance on terminal wealth, in which the interaction occurs by peer competition. In the model with n agents, the price dynamics of underlying risky assets depend on a common noise and contagious jump risk modelled by a multi-dimensional nonlinear Hawkes process. With a continuum of agents, we formulate the MFG problem and characterize a deterministic mean field equilibrium in an analytical form under some conditions, allowing us to investigate some impacts of model parameters in the limiting model and disc…
Mean-field quadratic BSDEs and related mean-field portfolio games of controls
Research Center for Mathematics and Interdisciplinary Sciences, Shandong University（山东大学数学与交叉科学研究院）；Department of Mathematics, Humboldt University Berlin（柏林洪堡大学数学系）
This paper is concerned with a linear quadratic (LQ, for short) optimal control problem for mean-field backward stochastic differential equations (MF-BSDE, for short) driven by a Poisson random martingale measure and a Brownian motion. Firstly, by the classic convex variation principle, the existence and uniqueness of the optimal control is established. Secondly, the optimal control is characterized by the stochastic Hamilton system which turns out to be a linear fully coupled mean-field forward-backward stochastic differential equation with jumps by the duality method. Thirdly, in terms of a …
In this paper, we study a linear-quadratic optimal control problem for mean-field stochastic differential equations driven by a Poisson random martingale measure and a multidimensional Brownian motion. Firstly, the existence and uniqueness of the optimal control is obtained by the classic convex variation principle. Secondly, by the duality method, the optimality system, also called the stochastic Hamilton system which turns out to be a linear fully coupled mean-field forward-backward stochastic differential equation with jumps, is derived to characterize the optimal control. Thirdly, applying…
On explicit solutions to a class of quadratic BSDEJs driven by affine Volterra processes with jumps and applications
Laboratoire de Probabilités, Statistique et Modélisation, UMR 8001, Sorbonne Université and Université Paris Cité（概率、统计与建模实验室，UMR 8001，索邦大学与巴黎西岱大学）；BPCE SA（BPCE股份公司）
In this paper, we study a stochastic linear-quadratic control problem with random coefficients and regime switching on a horizon $[0,T\wedgeτ]$, where $τ$ is a given random jump time for the underlying state process and $T$ is a constant. We obtain an explicit optimal state feedback control and explicit optimal cost value by solving a system of stochastic Riccati equations (SREs) with jumps on $[0,T\wedgeτ]$. By the decomposition approach stemming from filtration enlargement theory, we express the solution of the system of SREs with jumps in terms of another system of SREs involving only Brown…
Indefinite Stochastic Linear-Quadratic Optimal Control Problems with Random Coefficients and Poisson Jumps: Closed-Loop Representation of Open-Loop Optimal Controls
Southern University of Science and Technology（南方科技大学）；Southeast University（东南大学）
This paper considers linear-quadratic (LQ) stochastic leader-follower Stackelberg differential games for jump-diffusion systems with random coefficients. We first solve the LQ problem of the follower using the stochastic maximum principle and obtain the state-feedback representation of the open-loop optimal solution in terms of the integro-stochastic Riccati differential equation (ISRDE), where the state-feedback type control is shown to be optimal via the completion of squares method. Next, we establish the stochastic maximum principle for the indefinite LQ stochastic optimal control problem …
In this paper, we consider a linear-quadratic optimal control problem of mean-field stochastic differential equation with jump diffusion, which is also called as an MF-LQJ problem. Here, cost functional is allowed to be indefinite. We use an equivalent cost functional method to deal with the MF-LQJ problem with indefinite weighting matrices. Some equivalent cost functionals enable us to establish a bridge between indefinite and positive-definite MF-LQJ problems. With such a bridge, solvabilities of stochastic Hamiltonian system and Riccati equations are further characterized. Optimal control o…
This paper mainly investigates the optimal control and stabilization problems for linear discrete-time Markov jump systems. The general case for the finite-horizon optimal controller is considered, where the input weighting matrix in the performance index is just required to be positive semi-definite. The necessary and sufficient condition for the existence of the optimal controller in finite-horizon is given explicitly from a set of coupled difference Riccati equations (CDRE). One of the key techniques is to solve the forward and backward stochastic difference equation (FDSDE) which is obtain…
We study a general linear quadratic mean field type control problem and connect it to mean field games of a similar type. The solution is given both in terms of a forward/backward system of stochastic differential equations and by a pair of Riccati equations. In certain cases, the solution to the mean field type control is also the equilibrium strategy for a class of mean field games. We use this fact to study an economic model of production of exhaustible resources. Keywords: mean field type control, mean field games, linear-quadratic, optimal control, riccati equations, exhaustible resource …
This article examines mean-field-type game problems by means of a direct method. We provide various solvable examples beyond the classical linear-quadratic game problems. These include quadratic-quadratic games and games with power, logarithmic, sine square, hyperbolic sine square payoffs. Non-linear state dynamics such as log-state, control-dependent regime switching, quadratic state, cotangent state and hyperbolic cotangent state are considered. We identify equilibrium strategies and equilibrium payoffs in state-and-conditional mean-field type feedback form. It is shown that a simple direct …
We consider stochastic differential games with $N$ nearly identical players, linear-Gaussian dynamics, and infinite horizon discounted quadratic cost. Admissible controls are feedbacks for which the system is ergodic. We first study the existence of affine Nash equilibria by means of an associated system of $N$ Hamilton-Jacobi-Bellman and $N$ Kolmogorov-Fokker-Planck partial differential equations, proving that for small discount factors quadratic-Gaussian solutions exist and are unique. Then, we prove the convergence of such solutions to the unique quadratic-Gaussian solution of the pair of M…
This paper studies an asymptotic solvability problem for linear quadratic (LQ) mean field games with controlled diffusions and indefinite weights for the state and control in the costs. We employ a rescaling approach to derive a low dimensional Riccati ordinary differential equation (ODE) system, which characterizes a necessary and sufficient condition for asymptotic solvability. The rescaling technique is further used for performance estimates, establishing an $O(1/N)$-Nash equilibrium for the obtained decentralized strategies.
This paper focuses on linear-quadratic (LQ for short) mean-field games described by forward-backward stochastic differential equations (FBSDEs for short), in which the individual control region is postulated to be convex. The decentralized strategies and consistency condition are represented by a kind of coupled mean-field FBSDEs with projection operators. The well-posedness of consistency condition system is obtained using the monotonicity condition method. The $ε$-Nash equilibrium property is discussed as well.
We consider a controlled linear-quadratic (LQ) large-population system with mixture of three types agents: major leader, minor leaders and minor followers. The Stackelberg-Nash-Cournot (SNC) approximate equilibrium is studied by a major-minor mean-field game (MFG) coupled with a leader-follower Stackelberg game. By variational method, the SNC approximate equilibrium strategy can be represented by some forward-backward-stochastic-differential-equations (FBSDEs) in the open-loop sense. And we pay great effort to give the feedback form of the open-loop strategy by some Riccati equations.
We consider a Mean Field Games model where the dynamics of the agents is given by a controlled Langevin equation and the cost is quadratic. A change of variables, introduced in [9], transforms the Mean Field Games system into a system of two coupled kinetic Fokker-Planck equations. We prove an existence result for the latter system, obtaining consequently existence of a solution for the Mean Field Games system.
Owing to the growth of interest in Reinforcement Learning in the last few years, gradient based policy control methods have been gaining popularity for Control problems as well. And rightly so, since gradient policy methods have the advantage of optimizing a metric of interest in an end-to-end manner, along with being relatively easy to implement without complete knowledge of the underlying system. In this paper, we study the global convergence of gradient-based policy optimization methods for quadratic control of discrete-time and model-free Markovian jump linear systems (MJLS). We surmount m…
This paper is concerned with a linear quadratic stochastic Stackelberg differential game with time delay. The model is general, in which the state delay and the control delay both appear in the state equation, moreover, they both enter into the diffusion term. By introducing two Pseudo-Riccati equations and a special matrix equation, the state feedback representation of the open-loop Stackelberg strategy is derived, under some assumptions. Finally, two examples are given to illustrate the applications of the theoretical results.
This paper is concerned with a linear quadratic stochastic two-person zero-sum differential game with constant coefficients in an infinite time horizon. Open-loop and closed-loop saddle points are introduced. The existence of closed-loop saddle points is characterized by the solvability of an algebraic Riccati equation with a certain stabilizing condition. A crucial result makes our approach work is the unique solvability of a class of linear backward stochastic differential equations in an infinite horizon.
Mean field games with a major player were introduced in (Huang, 2010) within a linear-quadratic (LQ) modeling framework. Due to the rich structure of major-minor player models, the past ten years have seen significant research efforts for different solution notions and analytical techniques. For LQ models, we address the relation between three solution frameworks: the Nash certainty equivalence (NCE) approach in (Huang, 2010), master equations, and asymptotic solvability, which have been developed starting with different ideas. We establish their equivalence relationships.
This paper focuses on a kind of linear quadratic non-zero sum differential game driven by backward stochastic differential equation with asymmetric information, which is a natural continuation of Wang and Yu [IEEE TAC (2010) 55: 1742-1747, Automatica (2012) 48: 342-352]. Different from Wang and Yu [IEEE TAC (2010) 55: 1742-1747, Automatica (2012) 48: 342-352], novel motivations for studying this kind of game are provided. Some feedback Nash equilibrium points are uniquely obtained by forward-backward stochastic differential equations, their filters and the corresponding Riccati equations with …
In this paper, we study a class of linear-quadratic (LQ) mean field games of controls with common noises and their corresponding $N$-player games. The theory of mean field game of controls considers a class of mean field games where the interaction is via the joint law of both the state and control. By the stochastic maximum principle, we first analyze the limiting behavior of the representative player and obtain his/her optimal control in a feedback form with the given distributional flow of the population and its control. The mean field equilibrium is determined by the Nash certainty equival…
This paper investigates the so-called asymptotic solvability problem in linear quadratic (LQ) mean field games. The model has asymptotic solvability if for all sufficiently large population sizes, the corresponding game has a set of feedback Nash strategies subject to a mild regularity requirement. We provide a necessary and sufficient condition and show that in this case the solution converges to a mean field limit. This is accomplished by developing a re-scaling method to derive a low dimensional ordinary differential equation (ODE) system, where a non-symmetric Riccati ODE has a central rol…
