The paper is concerned with a zero-sum continuous-time stochastic differential game with a dynamics controlled by a Markov process and a terminal payoff. The value function of the original game is estimated using the value function of a model game. The dynamics of the model game differs from the original one. The general result applied to differential games yields the approximation of value function of differential game by the solution of countable system of ODEs.
We analyze a zero-sum stochastic differential game between two competing players who can choose unbounded controls. The payoffs of the game are defined through backward stochastic differential equations. We prove that each player's priority value satisfies a weak dynamic programming principle and thus solves the associated fully non-linear partial differential equation in the viscosity sense.
In this Note, assuming that the generator is uniform Lipschitz in the unknown variables, we relate the solution of a one dimensional backward stochastic differential equation with the value process of a stochastic differential game. Under a domination condition, a filtration-consistent evaluations is also related to a stochastic differential game. This relation comes out of a min-max representation for uniform Lipschitz functions as affine functions. The extension to reflected backward stochastic differential equations is also included.
We show that every two-player stochastic game with finite state and action sets and bounded, Borel-measurable, and shift-invariant payoffs, admits an $\ep$-equilibrium for all $\varepsilon>0$.
In this paper, we investigate Nash equilibrium payoffs for nonzero-sum stochastic differential games with reflection. We obtain an existence theorem and a characterization theorem of Nash equilibrium payoffs for nonzero-sum stochastic differential games with nonlinear cost functionals defined by doubly controlled reflected backward stochastic differential equations.
We establish regularity for functions satisfying a dynamic programming equation, which may arise for example from stochastic games or discretization schemes. Our results can also be utilized in obtaining regularity and existence results for the corresponding partial differential equations.
This paper is related to nonzero-sum stochastic differential games in the Markovian framework. We show existence of a Nash equilibrium point for the game when the drift is no longer bounded and only satisfies a linear growth condition. The main tool is the notion of backward stochastic differential equations which, in our case, are multidimensional with continuous coefficient and stochastic linear growth.
Shapley (1953) introduced two-player zero-sum discounted stochastic games, henceforth stochastic games, a model where a state variable follows a two-controlled Markov chain, the players receive rewards at each stage which add up to $0$, and each maximizes the normalized $\la$-discounted sum of stage rewards, for some fixed discount rate $\la\in(0,1]$. In this paper, we study asymptotic occupation measures arising in these games, as the discount rate goes to $0$.
In this paper, we study Nash equilibrium payoffs for nonzero-sum stochastic differential games via the theory of backward stochastic differential equations. We obtain an existence theorem and a characterization theorem of Nash equilibrium payoffs for nonzero-sum stochastic differential games with nonlinear cost functionals defined with the help of a doubly controlled backward stochastic differential equation. Our results extend former ones by Buckdahn, Cardaliaguet and Rainer (2004) and are based on a backward stochastic differential equation approach.
This work establishes sufficient conditions for existence of saddle points in discrete Markov games. The result reveals the relation between dynamic games and static games using dynamic programming equations. This result enables us to prove existence of saddle points of non-separable stochastic differential games of regime-switching diffusions under appropriate conditions.
We study a stochastic differential game between two players, controlling a forward stochastic Volterra integral equation (FSVIE). Each player has to optimize his own performance functional which includes a backward stochastic differential equation (BSDE). The dynamics considered are driven by time-changed Lévy noises, with absolutely continuous time-change process. We prove a sufficient maximum principle to characterize Nash equilibria and the related optimal strategies. For this we use techniques of control under partial information, and the non-anticipating stochastic derivative. The zero-su…
We introduce a new notion of pathwise strategies for stochastic differential games. This allows us to give a correct meaning to some statement asserted in [Cardaliaguet-Rainer 2009].
We consider a general class of nonzero-sum $N$-player stochastic games with impulse controls, where players control the underlying dynamics with discrete interventions. We adopt a verification approach and provide sufficient conditions for the Nash equilibria (NEs) of the game. We then consider the limit situation of $N \to \infty$, that is, a suitable mean-field game (MFG) with impulse controls. We show that under appropriate technical conditions, the existence of unique NE solution to the MFG, which is an $ε$-NE approximation to the $N$-player game, with $ε=O\left(\frac{1}{\sqrt{N}}\right)$.…
We consider a zero-sum stochastic game for continuous-time Markov chain with countable state space and unbounded transition and pay-off rates. The additional feature of the game is that the controllers together with taking actions are also allowed to stop the process. Under suitable hypothesis we show that the game has a value and it is the unique solution of certain dynamic programming inequalities with bilateral constraints. In the process we also prescribe a saddle point equilibrium.
We consider static finite-player network games and their continuum analogs, graphon games. Existence and uniqueness results are provided, as well as convergence of the finite-player network game optimal strategy profiles to their analogs for the graphon games. We also show that equilibrium strategy profiles of a graphon game provide approximate Nash equilibria for the finite-player games. Connections with mean field games and central planner optimization problems are discussed. Motivating applications are presented and explicit computations of their Nash equilibria and social optimal strategie…
We establish existence of nearly-optimal controls, conditions for existence of an optimal control and a saddle-point for respectively a control problem and zero-sum differential game associated with payoff functionals of mean-field type, under dynamics driven by weak solutions of stochastic differential equations of mean-field type.
This paper is devoted to a stochastic differential game of functional forward-backward stochastic differential equation (FBSDE, for short). The associated upper and lower value functions of the stochastic differential game are defined by controlled functional backward stochastic differential equations (BSDEs, for short). Applying the Girsanov transformation method introduced by Buckdahn and Li [1], the upper and the lower value functions are shown to be deterministic. We also generalize the Hamilton-Jacobi-Bellman-Isaacs (HJBI, for short) equations to the path-dependent ones. By establishing t…
In Gapeev and Kühn (2005), the stochastic game corresponding to perpetual convertible bonds was considered when driven by a Brownian motion and a compound Poisson process with exponential jumps. We consider the same stochastic game but driven by a spectrally positive Lévy process. We establish a complete solution to the game indicating four principle parameter regimes as well as characterizing the occurence of continuous and smooth fit. In Gapeev and Kühn (2005), the method of proof was mainly based on solving a free boundary value problem. In this paper, we instead use fluctuation theory and …
Zero sum games with risk-sensitive cost criterion are considered with underlying dynamics being given by controlled stochastic differential equations. Under the assumption of geometric stability on the dynamics , we completely characterize all possible saddle point strategies in the class of stationary Markov controls. In addition, we also establish existence-uniqueness result for the value function of the Hamilton-Jacobi-Isaacs equation.
This paper deals with a standard stochastic game model with a continuum of states under the duel-type setup. It newly proposes a hybrid model of game theory and the fluctuation process, which could be applied for various practical decision making situations. The unique theoretical stochastic game model is targeted to analyze a two-person duel-type game in the time domain. The parameters for strategic decisions including the moments of crossings, prior crossings, and the optimal number of iterations to get the highest winning chance are obtained by the compact closed joint functional. This pape…
In this paper, we consider the functional Itô calculus framework to find a path-dependent version of the Hamilton-Jacobi-Bellman equation for stochastic control problems that feature dynamics and running cost that depend on the path of the control. We also prove a Dynamic Programming Principle for such problems. We apply our results to path-dependence of the delay type. We further study Stochastic Differential Games in this context.
We introduce a notion of subgames for stochastic timing games and the related notion of subgame-perfect equilibrium in possibly mixed strategies. While a good notion of subgame-perfect equilibrium for continuous-time games is not available in general, we argue that our model is the appropriate version for timing games. We show that the notion coincides with the usual one for discrete-time games. Many timing games in continuous time have only equilibria in mixed strategies -- in particular preemption games, which often occur in the strategic real option literature. We provide a sound foundation…
This paper studies the mixed zero-sum stochastic differential game problem. We allow the functionals and dynamics to be of polynomial growth. The problem is formulated as an extended doubly reflected BSDEs with a specific generator. We show the existence of solution for this doubly reflected BSDEs and we prove the existence of a saddle-point of the game. Moreover, in the Markovian framework we prove that the value function is the unique viscosity solution of the associated Hamilton-Jacobi-Bellman equation.
The existence of stationary Markov perfect equilibria in stochastic games is shown under a general condition called "(decomposable) coarser transition kernels". This result covers various earlier existence results on correlated equilibria, noisy stochastic games, stochastic games with finite actions and state-independent transitions, and stochastic games with mixtures of constant transition kernels as special cases. A remarkably simple proof is provided via establishing a new connection between stochastic games and conditional expectations of correspondences. New applications of stochastic gam…
We consider multiplayer stochastic games in which the payoff of each player is a bounded and Borel-measurable function of the infinite play. By using a generalization of the technique of Martin (1998) and Maitra and Sudderth (1998), we show four different existence results. In each stochastic game, it holds for every $ε>0$ that (i) each player has a strategy that guarantees in each subgame that this player's payoff is at least her maxmin value up to $ε$, (ii) there exists a strategy profile under which in each subgame each player's payoff is at least her minmax value up to $ε$, (iii) the game …
A zero-sum differential game with controlled jump-diffusion driven state is considered, and studied using a combination of dynamic programming and viscosity solution techniques. We prove, under certain conditions, that the value of the game exists and is the unique viscosity solution of a fully nonlinear integro-partial differential equation. In addition, we formulate and prove a verification theorem for such games within the viscosity solution framework for nonlocal equations
The purpose of this paper is to study 2-person zero-sum stochastic differential games, in which one player is a major one and the other player is a group of $N$ minor agents which are collectively playing, statistically identical and have the same cost-functional. The game is studied in a weak formulation; this means in particular, we can study it as a game of the type "feedback control against feedback control". The payoff/cost functional is defined through a controlled backward stochastic differential equation, for which driving coefficient is assumed to satisfy strict concavity-convexity wi…
Zero-sum stochastic games generalize the notion of Markov Decision Processes (i.e. controlled Markov chains, or stochastic dynamic programming) to the 2-player competitive case : two players jointly control the evolution of a state variable, and have opposite interests. These notes constitute a short mathematical introduction to the theory of such games. Section 1 presents the basic model with finitely many states and actions. We give proofs of the standard results concerning : the existence and formulas for the values of the n-stage games, of the $λ$-discounted games, the convergence of these…
We adapt the Stochastic Perron's method in Bayraktar and Sirbu (ArXiv: 1103.0538) to the case of double obstacle problems associated to Dynkin games. We construct, symmetrically, a viscosity sub-solution which dominates the upper value of the game and a viscosity super-solution lying below the lower value of the game. If the double obstacle problem satisfies the viscosity comparison property, then the game has a value which is equal to the unique and continuous viscosity solution. In addition, the optimal strategies of the two players are equal to the first hitting times of the two stopping re…
We consider zero-sum stochastic differential games with possibly path-dependent controlled state. Unlike the previous literature, we allow for weak solutions of the state equation so that the players' controls are automatically of feedback type. Under some restrictions, needed for the a priori regularity of the upper and lower value functions of the game, we show that the game value exists when both the appropriate path-dependent Isaacs condition, and the uniqueness of viscosity solutions of the corresponding path-dependent Isaacs-HJB equation hold. We also provide a general verification argum…
We establish, for the first time, a connection between stochastic games and multiparameter eigenvalue problems, using the theory developed by Shapley and Snow (1950). This connection provides new results, new proofs, and new tools for studying stochastic games.
In this paper, we consider a partial information two-person zero-sum stochastic differential game problem where the system is governed by a backward stochastic differential equation driven by Teugels martingales associated with a Lévy process and an independent Brownian motion. One sufficient (a verification theorem) and one necessary conditions for the existence of optimal controls are proved. To illustrate the general results, a linear quadratic stochastic differential game problem is discussed.
We consider discrete time partially observable zero-sum stochastic game with average payoff criterion. We study the game using an equivalent completely observable game. We show that the game has a value and also we come up with a pair of optimal strategies for both the players.
In this paper, we present an optimal control problem for stochastic differential games under Markov regime-switching forward-backward stochastic differential equations with jumps and partial information. First, we prove a sufficient maximum principle for non zero-sum stochastic differential game problems and obtain equilibrium point for such games. Second, we prove an equivalent maximum principle for non zero-sum stochastic differential games. The zero-sum stochastic differential games equivalent maximum principle is then obtained as a corollary. We apply the obtained results to study a proble…
In this paper, we consider constrained discounted stochastic games with a countably generated state space and norm continuous transition probability having a density function. We prove existence of approximate stationary equilibria and stationary weak correlated equilibria. Our results imply the existence of stationary Nash equilibrium in $ARAT$ stochastic games.
We provide a direct, elementary proof for the existence of $\lim_{λ\to 0} v_λ$, where $v_λ$ is the value of a $λ$-discounted finite two-person zero-sum stochastic game.
We investigate a two-player zero-sum stochastic differential game in which the players have an asymmetric information on the random payoff. We prove that the game has a value and characterize this value in terms of dual solutions of some second order Hamilton-Jacobi equation.
This paper presents a new solution concept for multiplayer stochastic games, namely, acceptable strategy profiles. For each player $i$ and state $s$ in a stochastic game, let $w_i(s)$ be a real number. A strategy profile is \emph{$w$-acceptable}, where $w=(w_i(s))$, if the discounted payoff to each player $i$ at every initial state $s$ is at least $w_i(s)$, provided the discount factor of the players is sufficiently close to 1. Our goal is to provide simple strategy profiles that are $w$-acceptable for payoff vectors $w$ in which all coordinates are high.
In this paper, we consider a zero-sum undiscounted stochastic game which has finite state space and finitely many pure actions. Also, we assume the transition probability of the undiscounted stochastic game is controlled by one player and all the optimal strategies of the game are strictly positive. Under all the above assumptions, we show that the $β$-discounted stochastic games with the same payoff matrices and $β$ sufficiently close to 1 are also completely mixed. We give a counterexample to show that the converse of the above result in not true. We also show that, if we have non-zero value…
We consider both discrete and continuous time finite state-action stochastic games. In discrete time stochastic games, it is known that a stationary Blackwell-Nash equilibrium (BNE) exists for a single controller additive reward (SC-AR) stochastic game which is a special case of a general stochastic game. We show that, in general, the additive reward condition is needed for the existence of a BNE. We give an example of a single controller stochastic game which does not satisfy additive reward condition. We show that this example does not have a stationary BNE. For a general discrete time disco…
We study a zero-sum stochastic differential switching game in infinite horizon. We prove the existence of the value of the game and characterize it as the unique viscosity solution of the associated system of quasi-variational inequalities with bilateral obstacles. We also obtain a verification theorem which provides an optimal strategy of the game. Finally, some numerical examples with two regimes are given.
We develop an approach for two player constraint zero-sum and nonzero-sum stochastic differential games, which are modeled by Markov regime-switching jump-diffusion processes. We provide the relations between a usual stochastic optimal control setting and a Lagrangian method. In this context, we prove corresponding theorems for two different type of constraints, which lead us to find real valued and stochastic Lagrange multipliers, respectively. Then, we illustrate our results for a nonzero-sum game problem with stochastic maximum principle technique. Our application is an example of cooperati…
We study optimal behavior of energy producers under a CO_2 emission abatement program. We focus on a two-player discrete-time model where each producer is sequentially optimizing her emission and production schedules. The game-theoretic aspect is captured through a reduced-form price-impact model for the CO_2 allowance price. Such duopolistic competition results in a new type of a non-zero-sum stochastic switching game on finite horizon. Existence of game Nash equilibria is established through generalization to randomized switching strategies. No uniqueness is possible and we therefore conside…
We study stochastic differential games of jump diffusions, where the players have access to inside information. Our approach is based on anticipative stochastic calculus, white noise, Hida-Malliavin calculus, forward integrals and the Donsker delta functional. We obtain a characterization of Nash equilibria of such games in terms of the corresponding Hamiltonians. This is used to study applications to insider games in finance, specifically optimal insider consumption and optimal insider portfolio under model uncertainty.
In this paper, an open-loop two-person non-zero sum stochastic differential game is considered for forward-backward stochastic systems. More precisely, the controlled systems are described by a fully coupled nonlinear multi- dimensional forward-backward stochastic differential equation driven by a multi-dimensional Brownian motion. one sufficient (a verification theorem) and one necessary conditions for the existence of open-loop Nash equilibrium points for the corresponding two-person non-zero sum stochastic differential game are proved. The control domain need to be convex and the admissible…
Definable zero-sum stochastic games involve a finite number of states and action sets, reward and transition functions that are definable in an o-minimal structure. Prominent examples of such games are finite, semi-algebraic or globally subanalytic stochastic games. We prove that the Shapley operator of any definable stochastic game with separable transition and reward functions is definable in the same structure. Definability in the same structure does not hold systematically: we provide a counterexample of a stochastic game with semi-algebraic data yielding a non semi-algebraic but globally …
This article is related to risk-sensitive nonzero-sum stochastic differential games in the Markovian framework. This game takes into account the attitudes of the players toward risk and the utility is of exponential form. We show the existence of a Nash equilibrium point for the game when the drift is no longer bounded and only satisfies a linear growth condition. The main tool is the notion of backward stochastic differential equation, which in our case, is multidimensional with continuous generator involving both a quadratic term of Z and a stochastic linear growth component with respect to …
We consider a finite state, finite action, zero-sum stochastic games with data defining the game lying in the ordered field of algebraic numbers. In both the discounted and the limiting average versions of these games we prove that the value vector also lies in the same field of algebraic numbers. In a prescribed sense, our results settle a problem that has remained open since, at least, 1991.
We extend the stochastic Perron method to analyze the framework of stochastic target games, in which one player tries to find a strategy such that the state process almost surely reaches a given target no matter which action is chosen by the other player. Within this framework, our method produces a viscosity sub-solution (super-solution) of a Hamilton-Jacobi-Bellman (HJB) equation. We then characterize the value function as a viscosity solution to the HJB equation using a comparison result and a byproduct to obtain the dynamic programming principle.
We study two classes of zero-sum stochastic games with compact action sets and a finite product state space. These two classes assume a communication property on the state spaces of the players. For strongly communicating on one side games, we prove the existence of the uniform value. For weakly communicating on both sides games, we prove that the asymptotic value, and therefore the uniform value, may fail to exist.
This paper proves several Tauberian theorems for general iterations of operators, and provides two applications to zero-sum stochastic games where the total payoff is a weighted sum of the stage payoffs. The first application is to provide conditions under which the existence of the asymptotic value implies the convergence of the values of the weighted game, as players get more and more patient. The second application concerns stochastic games with finite state space and action sets. This paper builds a simple class of asymptotically optimal strategies in the weighted game, that at each stage …
Stochastic games are an important class of problems that generalize Markov decision processes to game theoretic scenarios. We consider finite state two-player zero-sum stochastic games over an infinite time horizon with discounted rewards. The players are assumed to have infinite strategy spaces and the payoffs are assumed to be polynomials. In this paper we restrict our attention to a special class of games for which the single-controller assumption holds. It is shown that minimax equilibria and optimal strategies for such games may be obtained via semidefinite programming.
