We consider a general class of nonzero-sum $N$-player stochastic games with impulse controls, where players control the underlying dynamics with discrete interventions. We adopt a verification approach and provide sufficient conditions for the Nash equilibria (NEs) of the game. We then consider the limit situation of $N \to \infty$, that is, a suitable mean-field game (MFG) with impulse controls. We show that under appropriate technical conditions, the existence of unique NE solution to the MFG, which is an $ε$-NE approximation to the $N$-player game, with $ε=O\left(\frac{1}{\sqrt{N}}\right)$. As an example, we analyze in details a class of two-player stochastic games which extends the classical cash management problem to the game setting. In particular, we present numerical analysis for the cases of the single player, the two-player game, and the MFG, showing the impact of competition on the player's optimal strategy, with sensitivity analysis of the model parameters.
We study a two-player nonzero-sum stochastic differential game where one player controls the state variable via additive impulses while the other player can stop the game at any time. The main goal of this work is characterize Nash equilibria through a verification theorem, which identifies a new system of quasi-variational inequalities whose solution gives equilibrium payoffs with the correspondent strategies. Moreover, we apply the verification theorem to a game with a one-dimensional state variable, evolving as a scaled Brownian motion, and with linear payoff and costs for both players. Two…
This paper considers the problem of two-player zero-sum stochastic differential game with both players adopting impulse controls in finite horizon under rather weak assumptions on the cost functions ($c$ and $χ$ not decreasing in time). We use the dynamic programming principle and viscosity solutions approach to show existence and uniqueness of a solution for the Hamilton-Jacobi-Bellman-Isaacs (HJBI) partial differential equation (PDE) of the game. We prove that the upper and lower value functions coincide.
We consider a two-player linear-state differential game, where one player intervenes continuously in the game, while the other implements an impulse control. When the impulse instants are exogenous, we obtain the classical result in linear-state differential games that open-loop and feedback Nash equilibria coincide. When the impulse instants are endogenous, we show that this result no longer holds. In particular, the two equilibria are different. We show that the impulse level is a constant in both equilibria. More importantly, in the open-loop case, we show that the equilibrium number of imp…
We consider a two-player zero-sum deterministic differential game where each player uses both continuous and impulse controls in infinite-time horizon. We assume that the impulses supposed to be of general term and the costs depend on the state of the system. We use the dynamic programming principle and viscosity solutions approach to show existence and uniqueness of a solution for the Hamilton-Jacobi-Bellman-Isaacs (HJBI) partial differential equations (PDEs) of the game. We prove under Isaacs condition that the upper and lower value functions coincide.
In the nonzero-sum setting, we establish a connection between Nash equilibria in games of optimal stopping (Dynkin games) and generalised Nash equilibrium problems (GNEP). In the Dynkin game this reveals novel equilibria of threshold type and of more complex types, and leads to novel uniqueness and stability results.
This paper focuses on zero-sum stochastic differential games in the framework of forward-backward stochastic differential equations on a finite time horizon with both players adopting impulse controls. By means of BSDE methods, in particular that of the notion from Peng's stochastic \textit{% backward semigroups}, we prove a dynamic programming principle for both the upper and the lower value functions of the game. The upper and the lower value functions are then shown to be the unique viscosity solutions of the Hamilton-Jacobi-Bellman-Isaacs equations with a double-obstacle. As a consequence,…
In the present paper, we study a two-player zero-sum deterministic differential game with both players adopting impulse controls, in infinite time horizon, under rather weak assumptions on the cost functions. We prove by means of the dynamic programming principle (DPP) that the lower and upper value functions are continuous and viscosity solutions to the corresponding Hamilton-Jacobi-Bellman-Isaacs (HJBI) quasi-variational inequality (QVI). We define a new HJBI QVI for which, under a proportional property assumption on the maximizer cost, the value functions are the unique viscosity solution. …
This paper is related to nonzero-sum stochastic differential games in the Markovian framework. We show existence of a Nash equilibrium point for the game when the drift is no longer bounded and only satisfies a linear growth condition. The main tool is the notion of backward stochastic differential equations which, in our case, are multidimensional with continuous coefficient and stochastic linear growth.
We study a nonzero-sum stochastic differential game with both players adopting impulse controls, on a finite time horizon. The objective of each player is to maximize her total expected discounted profits. The resolution methodology relies on the connection between Nash equilibrium and the corresponding system of quasi-variational inequalities (QVIs in short). We prove, by means of the weak dynamic programming principle for the stochastic differential game, that the value function of each player is a constrained viscosity solution to the associated QVIs system in the class of linear growth fun…
We consider a stochastic differential game in the context of forward-backward stochastic differential equations, where one player implements an impulse control while the opponent controls the system continuously. Utilizing the notion of "backward semigroups" we first prove the dynamic programming principle (DPP) for a truncated version of the problem in a straightforward manner. Relying on a uniform convergence argument then enables us to show the DPP for the general setting. In particular, this avoids technical constraints imposed in previous works dealing with the same problem. Moreover, our…
A policy iteration algorithm for nonzero-sum stochastic impulse games
London School of Economics and Political Science（伦敦政治经济学院）；Université Paris Dauphine, PSL Research University（巴黎第九大学，PSL研究大学）；Ecole Polytechnique（巴黎综合理工学院）；University of Tunis El Manar（突尼斯埃尔马纳尔大学）；IMPA（巴西国家数学研究所）
We study the infinite horizon discrete time N-player nonzero-sum Dynkin game ($N \geq 2$) with stopping times as strategies (or pure strategies). We prove existence of an $\varepsilon$-Nash equilibrium point for the game by presenting a constructive algorithm. One of the main features is that the payoffs of the players depend on the set of players that stop at the termination stage which is the minimal stage in which at least one player stops. The existence result is extended to the case of a nonzero-sum game with finite horizon. Finally, the algorithm is illustrated by two explicit examples i…
We consider an investment problem in which an investor performs capital injections to increase the liquidity of a firm for it to maximise profit from market operations. Each time the investor performs an injection, the investor incurs a fixed transaction cost. In addition to maximising their terminal reward, the investor seeks to minimise the risk of loss of their investment (from a possible firm ruin) by exiting the market at some point in time. We show that the problem can be reformulated in terms of a new stochastic differential game of control and stopping in which one of the players modif…
Nonzero-sum stochastic differential games with impulse controls offer a realistic and far-reaching modelling framework for applications within finance, energy markets, and other areas, but the difficulty in solving such problems has hindered their proliferation. Semi-analytical approaches make strong assumptions pertaining to very particular cases. To the author's best knowledge, the only numerical method in the literature is the heuristic one we put forward to solve an underlying system of quasi-variational inequalities. Focusing on symmetric games, this paper presents a simpler, more precise…
We study a new kind of non-zero-sum stochastic differential game with mixed impulse/switching controls, motivated by strategic competition in commodity markets. A representative upstream firm produces a commodity that is used by a representative downstream firm to produce a final consumption good. Both firms can influence the price of the commodity. By shutting down or increasing generation capacities, the upstream firm influences the price with impulses. By switching (or not) to a substitute, the downstream firm influences the drift of the commodity price process. We study the resulting impul…
In this paper we study the zero-sum and nonzero-sum differential games with not assuming Isaacs condition. Along with the partition $π$ of the time interval $[0,T]$, we choose the suitable random non-anticipative strategy with delay to study our differential games with asymmetric information. Using Fenchel transformation, we prove that the limits of the upper value function $W^π$ and lower value function $V^π$ coincide when the mesh of partition $π$ tends to 0. Moreover, we give a characterization for the Nash equilibrium payoff (NEP, for short) of our nonzero-sum differential games without Is…
This paper analyses two-player nonzero-sum games of optimal stopping on a class of linear regular diffusions with not non-singular boundary behaviour (in the sense of Itô and McKean (1974), p.\ 108). We provide sufficient conditions under which Nash equilibria are realised by each player stopping the diffusion at one of the two boundary points of an interval. The boundaries of this interval solve a system of algebraic equations. We also provide conditions sufficient for the uniqueness of the equilibrium in this class.
We consider a general type of non-Markovian impulse control problems under adverse non-linear expectation or, more specifically, the zero-sum game problem where the adversary player decides the probability measure. We show that the upper and lower value functions satisfy a dynamic programming principle (DPP). We first prove the dynamic programming principle (DPP) for a truncated version of the upper value function in a straightforward manner. Relying on a uniform convergence argument then enables us to show the DPP for the general setting. Following this, we use an approximation based on a com…
We construct an approximate public-signal correlated equilibrium for a nonzero-sum differential game in the class of stochastic strategies with memory. The construction is based on a solution of an auxiliary nonzero-sum continuous-time stochastic game. This class of games includes stochastic differential games and continuous-time Markov games. Moreover, we study the limit of approximate equilibrium outcomes in the case when the auxiliary stochastic games tend to the original deterministic one. We show that it lies in the convex hull of the set of equilibrium values provided by deterministic pu…
In this paper we establish a new connection between a class of 2-player nonzero-sum games of optimal stopping and certain $2$-player nonzero-sum games of singular control. We show that whenever a Nash equilibrium in the game of stopping is attained by hitting times at two separate boundaries, then such boundaries also trigger a Nash equilibrium in the game of singular control. Moreover a differential link between the players' value functions holds across the two games.
This paper considers a new class of deterministic finite-time horizon, two-player, zero-sum differential games (DGs) in which the maximizing player is allowed to take continuous and impulse controls whereas the minimizing player is allowed to take impulse control only. We seek to approximate the value function, and to provide a verification theorem for this class of DGs. We first, by means of dynamic programming principle (DPP) in viscosity solution (VS) framework, characterize the value function as the unique VS to the related Hamilton-Jacobi-Bellman-Isaacs (HJBI) double-obstacle equation. Ne…
In this paper, we investigate Nash equilibrium payoffs for nonzero-sum stochastic differential games with reflection. We obtain an existence theorem and a characterization theorem of Nash equilibrium payoffs for nonzero-sum stochastic differential games with nonlinear cost functionals defined by doubly controlled reflected backward stochastic differential equations.
We study two person nonzero-sum stochastic differential games with risk-sensitive discounted and ergodic cost criteria. Under certain conditions we establish a Nash equilibrium in Markov strategies for the discounted cost criterion and a Nash equilibrium in stationary strategies for the ergodic cost criterion. We achieve our results by studying the relevant systems of coupled HJB equations.
In this paper, we study a nonzero-sum stochastic differential game in Markovian framework. We show the existence of the Nash equilibrium point which is discontinuous and of bang-bang type under natural conditions. The main tool is the notion of backward stochastic differential equations which, in our case, are multidimensional with discontinuous generators with respect to z component.
This article is related to risk-sensitive nonzero-sum stochastic differential games in the Markovian framework. This game takes into account the attitudes of the players toward risk and the utility is of exponential form. We show the existence of a Nash equilibrium point for the game when the drift is no longer bounded and only satisfies a linear growth condition. The main tool is the notion of backward stochastic differential equation, which in our case, is multidimensional with continuous generator involving both a quadratic term of Z and a stochastic linear growth component with respect to …
The infinite horizon risk-sensitive discounted-cost and ergodic-cost nonzero-sum stochastic games for controlled Markov chains with countably many states are analyzed. For the discounted-cost game, we prove the existence of Nash equilibrium strategies in the class of Markov strategies under fairly general conditions. Under an additional geometric ergodicity condition and a small cost criterion, the existence of Nash equilibrium strategies in the class of stationary Markov strategies is proved for the ergodic-cost game.
We study nonzero-sum stochastic differential games with risk-sensitive ergodic cost criterion. Under certain conditions, using multi-parameter eigenvalue approach, we establish the existence of a Nash equilibrium in the space of stationary Markov strategies. We achieve our results by studying the relevant systems of coupled HJB equations. Exploiting the stochastic representation of the principal eigenfunctions we completely characterize Nash equilibrium points in the space of stationary Markov strategies.
In this paper, we study Nash equilibrium payoffs for nonzero-sum stochastic differential games via the theory of backward stochastic differential equations. We obtain an existence theorem and a characterization theorem of Nash equilibrium payoffs for nonzero-sum stochastic differential games with nonlinear cost functionals defined with the help of a doubly controlled backward stochastic differential equation. Our results extend former ones by Buckdahn, Cardaliaguet and Rainer (2004) and are based on a backward stochastic differential equation approach.
We study nonzero-sum stochastic games for continuous time Markov decision processes on a denumerable state space with risk-sensitive ergodic cost criterion. Transition rates and cost rates are allowed to be unbounded. Under a Lyapunov type stability assumption, we show that the corresponding system of coupled HJB equations admits a solution which leads to the existence of a Nash equilibrium in stationary strategies. We establish this using an approach involving principal eigenvalues associated with the HJB equations. Furthermore, exploiting appropriate stochastic representation of principal ei…
In this paper, the open-loop and closed-loop local and remote stochastic nonzero-sum game (LRSNG) problem is investigated. Different from previous works, the stochastic nonzero-sum game problem under consideration is essentially a special class of two-person nonzero-sum game problem, in which the information sets accessed by the two players are inconsistent. More specifically, both the local player and the remote player are involved in the system dynamics, and the information sets obtained by the two players are different, and each player is designed to minimize its own cost function. For the …
We study nonzero-sum stochastic games for continuous time Markov chains on a denumerable state space with risk sensitive discounted and ergodic cost criteria. For the discounted cost criterion we first show that the corresponding system of coupled HJB equations has an appropriate solution. Then under an additional additive structure on the transition rate matrix and payoff functions, we establish the existence of a Nash equilibrium in Markov strategies. For the ergodic cost criterion we assume a Lyapunov type stability assumption and a small cost condition. Under these assumptions we show that…
We consider a nonzero-sum Markov game on an abstract measurable state space with compact metric action spaces. The goal of each player is to maximize his respective discounted payoff function under the condition that some constraints on a discounted payoff are satisfied. We are interested in the existence of a Nash or noncooperative equilibrium. Under suitable conditions, which include absolute continuity of the transitions with respect to some reference probability measure, additivity of the payoffs and the transition probabilities (ARAT condition), and continuity in action of the payoff func…
Nonzero sum games typically have multiple Nash equilibriums (or no equilibrium), and unlike the zero sum case, they may have different values at different equilibriums. Instead of focusing on the existence of individual equilibriums, we study the set of values over all equilibriums, which we call the set value of the game. The set value is unique by nature and always exists (with possible value $\emptyset$). Similar to the standard value function in control literature, it enjoys many nice properties such as regularity, stability, and more importantly the dynamic programming principle. There ar…
We consider a class of N-player stochastic games of multi-dimensional singular control, in which each player faces a minimization problem of monotone-follower type with submodular costs. We call these games "monotone-follower games". In a not necessarily Markovian setting, we establish the existence of Nash equilibria. Moreover, we introduce a sequence of approximating games by restricting, for each natural number n, the players' admissible strategies to the set of Lipschitz processes with Lipschitz constant bounded by n. We prove that, for each n, there exists a Nash equilibrium of the approx…
We study a discrete-time finite-horizon two-players nonzero-sum stopping game where the filtration of Player 1 is richer than the filtration of Player 2. A major difficulty which is caused by the information asymmetry is that Player 2 may not know whether Player 1 has already stopped the game or not. Furthermore, the classical backward-induction approach is not applicable in the current setup. This is because when the informed player decides not to stop, he reveals information to the uninformed player and hence the decision of the uninformed player at time $t$ may not be determined by the play…
The topics treated in this thesis are inherently two-fold. The first part considers the problem of a market maker optimally setting bid/ask quotes over a finite time horizon, to maximize her expected utility. The intensities of the orders she receives depend not only on the spreads she quotes, but also on unobservable factors modelled by a hidden Markov chain. This stochastic control problem under partial information is solved by means of stochastic filtering, control and PDMPs theory. The value function is characterized as the unique continuous viscosity solution of its dynamic programming eq…
We study model-based and model-free policy optimization in a class of nonzero-sum stochastic dynamic games called linear quadratic (LQ) deep structured games. In such games, players interact with each other through a set of weighted averages (linear regressions) of the states and actions. In this paper, we focus our attention to homogeneous weights; however, for the special case of infinite population, the obtained results extend to asymptotically vanishing weights wherein the players learn the sequential weighted mean-field equilibrium. Despite the non-convexity of the optimization in policy …
In this paper, we consider a linear quadratic stochastic two-person nonzero-sum differential game. Open-loop and closed-loop Nash equilibria are introduced. The existence of the former is characterized by the solvability of a system of forward-backward stochastic differential equations, and that of the latter is characterized by the solvability of a system of coupled symmetric Riccati differential equations. Sometimes, open-loop Nash equilibria admit a closed-loop representation, via the solution to a system of non-symmetric Riccati equations, which is different from the outcome of the closed-…
A two-person zero-sum infinite dimensional differential game of infinite duration with discounted payoff involving hybrid controls is studied. The minimizing player is allowed to take continuous, switching and impulse controls whereas the maximizing player is allowed to take continuous and switching controls. By taking strategies in the sense of Elliott--Kalton, we prove the existence of value and characterize it as the unique viscosity solution of the associated system of quasi-variational inequalities.
We discuss a class of explicitly solvable mean field type control problems/mean field games with a clear economic interpretation. More precisely, we consider long term average impulse control problems with underlying general one-dimensional diffusion processes motivated by optimal harvesting problems in natural resource management. We extend the classical stochastic Faustmann models by allowing the prices to depend on the state of the market using a mean field structure. In a competitive market model, we prove that, under natural conditions, there exists an equilibrium strategy of threshold-ty…
We develop an approach for two player constraint zero-sum and nonzero-sum stochastic differential games, which are modeled by Markov regime-switching jump-diffusion processes. We provide the relations between a usual stochastic optimal control setting and a Lagrangian method. In this context, we prove corresponding theorems for two different type of constraints, which lead us to find real valued and stochastic Lagrange multipliers, respectively. Then, we illustrate our results for a nonzero-sum game problem with stochastic maximum principle technique. Our application is an example of cooperati…
This paper deals with an extension of the concept of correlated strategies to Markov stopping games. The Nash equilibrium approach to solving nonzero-sum stopping games may give multiple solutions. An arbitrator can suggest to each player the decision to be applied at each stage based on a joint distribution over the players' decisions. This is a form of equilibrium selection. Examples of correlated equilibria in nonzero-sum games related to the staff selection competition in the case of two departments are given. Utilitarian, egalitarian, republican and libertarian concepts of correlated equi…
This paper analyses a stochastic differential game of control and stopping in which one of the players modifies a diffusion process using impulse controls, an adversary then chooses a stopping time to end the game. The paper firstly establishes the regularity and boundedness of the upper and lower value functions from which an appropriate variant of the dynamic programming principle (DPP) is derived. It is then proven that the upper and lower value functions coincide so that the game admits a value and that the value of the game is a unique viscosity solution to a HJBI equation described by a …
In this paper, a Nash-type fictitious game framework is introduced to handle a time-inconsistent linear-quadratic optimal control. The Nash-type game in this framework is called fictitious as it is between the decision maker (called real player) and an auxiliary control variable (called fictitious player) with the real player and fictitious player looking for time-consistent policy and precomitted optimal policy, respectively. Namely, the fictitious-game framework is actually an auxiliary-variable-based mechanism where the fictitious player is our particular design. Noting that the real player…
Two-player mean-payoff Stackelberg games are nonzero-sum infinite duration games played on a bi-weighted graph by Leader (Player 0) and Follower (Player 1). Such games are played sequentially: first, Leader announces her strategy, second, Follower chooses his best-response. If we cannot impose which best-response is chosen by Follower, we say that Follower, though strategic, is adversarial towards Leader. The maximal value that Leader can get in this nonzero-sum game is called the adversarial Stackelberg value (ASV) of the game. We study the robustness of strategies for Leader in these games a…
