We develop the linear programming approach to mean-field games in a general setting. This relaxed control approach allows to prove existence results under weak assumptions, and lends itself well to numerical implementation. We consider mean-field game problems where the representative agent chooses both the optimal control and the optimal time to exit the game, where the instantaneous reward function and the coefficients of the state process may depend on the distribution of the other agents. Furthermore, we establish the equivalence between mean-field games equilibria obtained by the linear programming approach and the ones obtained via the controlled/stopped martingale approach, another relaxation method used in a few previous papers in the case when there is only control.
We consider multi-player stopping games in continuous time. Unlike Dynkin games, in our games the payoff of each player is revealed after all the players stop. Moreover, each player can adjust her own stopping strategy by observing other players' behaviors. Assuming the continuity of the payoff functions in time, we show that there always exists an $\eps$-Nash equilibrium in pure stopping strategies for any $\eps>0$.
In this paper we establish a new connection between a class of 2-player nonzero-sum games of optimal stopping and certain $2$-player nonzero-sum games of singular control. We show that whenever a Nash equilibrium in the game of stopping is attained by hitting times at two separate boundaries, then such boundaries also trigger a Nash equilibrium in the game of singular control. Moreover a differential link between the players' value functions holds across the two games.
The properties of value functions of time inhomogeneous optimal stopping problem and zero-sum game (Dynkin game) are studied through time dependent Dirichlet form. Under the absolute continuity condition on the transition function of the underlying diffusion process and some other assumptions, the refined solutions without exceptional starting points are proved to exist, and the value functions of the optimal stopping and zero-sum game, which are finely and cofinely continuous, are characterized as the solutions of some variational inequalities, respectively.
On a filtered probability space $(Ω,\mathcal{F}, (\mathcal{F}_t)_{t\in[0,\infty]}, \mathbb{P})$, we consider the two-player non-zero-sum stopping game $u^i := \mathbb{E}[U^i(ρ,τ)],\ i=1,2$, where the first player choose a stopping strategy $ρ$ to maximize $u^1$ and the second player chose a stopping strategy $τ$ to maximize $u^2$. Unlike the Dynkin game, here we assume that $U(s,t)$ is $\mathcal{F}_{s\vee t}$-measurable. Assuming the continuity of $U^i$ in $(s,t)$, we show that there exists an $ε$-Nash equilibrium for any $ε>0$.
This paper analyses a stochastic differential game of control and stopping in which one of the players modifies a diffusion process using impulse controls, an adversary then chooses a stopping time to end the game. The paper firstly establishes the regularity and boundedness of the upper and lower value functions from which an appropriate variant of the dynamic programming principle (DPP) is derived. It is then proven that the upper and lower value functions coincide so that the game admits a value and that the value of the game is a unique viscosity solution to a HJBI equation described by a …
We develop the fictitious play algorithm in the context of the linear programming approach for mean field games of optimal stopping and mean field games with regular control and absorption. This algorithm allows to approximate the mean field game population dynamics without computing the value function by solving linear programming problems associated with the distributions of the players still in the game and their stopping times/controls. We show the convergence of the algorithm using the topology of convergence in measure in the space of subprobability measures, which is needed to deal with…
We show that the value function of an optimal stopping game driven by a one-dimensional diffusion can be characterised using a modification of the Legendre transformation if and only if the optimal stopping game exhibits a Nash equilibrium (i.e. a saddle point of the optimal stopping game exists). This result is an analytical complement to the results in Peskir, G. (2012) A Duality Principle for the Legendre Transform. Journal of Convex Analysis, 19(3), 609-630 where the `duality' between a concave-biconjugate which is modified to remain below an upper barrier and a convex-biconjugate which is…
This paper studies a class of zero-sum stopping game in a regime switching model. A verification theorem as a sufficient criterion for Nash equilibriums is established based on a set of variational inequalities (VIs). Under an appropriate regularity condition for solutions to the VIs, a suitable system of algebraic equations is derived via the so-called smooth-fit principle. Explicit Nash equilibrium stopping rules of threshold-type for the two players and the corresponding value function of the game in closed form are obtained. Numerical experiments are reported to demonstrate the dependence …
This paper revisits the well-studied \emph{optimal stopping} problem but within the \emph{large-population} framework. In particular, two classes of optimal stopping problems are formulated by taking into account the \emph{relative performance criteria}. It is remarkable the relative performance criteria, also understood by the \emph{Joneses preference}, \emph{habit formation utility}, or \emph{relative wealth concern} in economics and finance, plays an important role in explaining various decision behaviors such as price bubbles. By introducing such criteria in large-population setting, a giv…
We study a zero-sum game where the evolution of a spectrally one-sided Levy process is modified by a singular controller and is terminated by the stopper. The singular controller minimizes the expected values of running, controlling and terminal costs while the stopper maximizes them. Using fluctuation theory and scale functions, we derive a saddle point and the value function of the game. Numerical examples under phase-type Levy processes are also given.
We consider an investment problem in which an investor performs capital injections to increase the liquidity of a firm for it to maximise profit from market operations. Each time the investor performs an injection, the investor incurs a fixed transaction cost. In addition to maximising their terminal reward, the investor seeks to minimise the risk of loss of their investment (from a possible firm ruin) by exiting the market at some point in time. We show that the problem can be reformulated in terms of a new stochastic differential game of control and stopping in which one of the players modif…
Bellman formulated a vague principle for optimization over time, which characterizes optimal policies by stating that a decision maker should not regret previous decisions retrospectively. This paper addresses time consistency in stochastic optimization. The problem is stated in generality first. The paper discusses time consistent decision-making by addressing risk measures which are recursive, nested, dynamically or time consistent and introduces stopping time risk measures. It turns out that the paradigm of time consistency is in conflict with various desirable, classical properties of gene…
We analyze an optimal stopping problem with a constraint on the expected cost. When the reward function and cost function are Lipschitz continuous in state variable, we show that the value of such an optimal stopping problem is a continuous function in current state and in budget level. Then we derive a dynamic programming principle (DPP) for the value function in which the conditional expected cost acts as an additional state process. As the optimal stopping problem with expectation constraint can be transformed to a stochastic optimization problem with supermartingale controls, we explore a …
We consider two-player non-zero-sum stopping games in discrete time. Unlike Dynkin games, in our games the payoff of each player is revealed after both players stop. Moreover, each player can adjust her own stopping strategy according to the other player's action. In the first part of the paper, we consider the game where players act simultaneously at each stage. We show that there exists a Nash equilibrium in mixed stopping strategies. In the second part, we assume that one player has to act first at each stage. In this case, we show the existence of a Nash equilibrium in pure stopping strate…
We study two-player zero-sum stopping games in continuous time and infinite horizon. We prove that the value in randomized stopping times exists as soon as the payoff processes are right-continuous. In particular, as opposed to existing literature, we do not assume any conditions on the relations between the payoff processes. We also show that both players have simple epsilon- optimal randomized stopping times; namely, randomized stopping times which are small perturbations of non-randomized stopping times.
In the nonzero-sum setting, we establish a connection between Nash equilibria in games of optimal stopping (Dynkin games) and generalised Nash equilibrium problems (GNEP). In the Dynkin game this reveals novel equilibria of threshold type and of more complex types, and leads to novel uniqueness and stability results.
We study a combined optimal control/stopping problem under a nonlinear expectation ${\cal E}^f$ induced by a BSDE with jumps, in a Markovian framework. The terminal reward function is only supposed to be Borelian. The value function $u$ associated with this problem is generally irregular. We first establish a {\em sub- (resp. super-) optimality principle of dynamic programming} involving its {\em upper- (resp. lower-) semicontinuous envelope} $u^*$ (resp. $u_*$). This result, called {\em weak} dynamic programming principle (DPP), extends that obtained in \cite{BT} in the case of a classical ex…
A game-theoretic framework for time-inconsistent stopping problems where the time-inconsistency is due to the consideration of a non-linear function of an expected reward is developed. A class of mixed strategy stopping times that allows the agents in the game to jointly choose the intensity function of a Cox process is introduced and motivated. A subgame perfect Nash equilibrium is defined. The equilibrium is characterized and other necessary and sufficient equilibrium conditions including a smooth fit result are proved. Existence and uniqueness are investigated. A mean-variance and a varianc…
We consider an optimal stopping problem where a constraint is placed on the distribution of the stopping time. Reformulating the problem in terms of so-called measure-valued martingales allows us to transform the marginal constraint into an initial condition and view the problem as a stochastic control problem; we establish the corresponding dynamic programming principle.
We consider a zero-sum continuous time stopping game in which the pay-off is revealed in the maximum of the two stopping times instead of the minimum, which is the case in Dynkin games.
We study a discrete-time finite-horizon two-players nonzero-sum stopping game where the filtration of Player 1 is richer than the filtration of Player 2. A major difficulty which is caused by the information asymmetry is that Player 2 may not know whether Player 1 has already stopped the game or not. Furthermore, the classical backward-induction approach is not applicable in the current setup. This is because when the informed player decides not to stop, he reveals information to the uninformed player and hence the decision of the uninformed player at time $t$ may not be determined by the play…
We study a model of two-player, zero-sum, stopping games with asymmetric information. We assume that the payoff depends on two continuous-time Markov chains (X, Y), where X is only observed by player 1 and Y only by player 2, implying that the players have access to stopping times with respect to different filtrations. We show the existence of a value in mixed stopping times and provide a variational characterization for the value as a function of the initial distribution of the Markov chains. We also prove a verification theorem for optimal stopping rules which allows to construct optimal sto…
We consider the mean-field game where each agent determines the optimal time to exit the game by solving an optimal stopping problem with reward function depending on the density of the state processes of agents still present in the game. We place ourselves in the framework of relaxed optimal stopping, which amounts to looking for the optimal occupation measure of the stopper rather than the optimal stopping time. This framework allows us to prove the existence of the relaxed Nash equilibrium and the uniqueness of the associated value of the representative agent under mild assumptions. Further…
The paper is concerned with a variant of the continuous-time finite state Markov game of control and stopping where both players can affect transition rates, while only one player can choose a stopping time. We use the dynamic programming principle and reduce this problem to a system of ODEs with unilateral constraints. This system plays the role of the Bellman equation. We show that its solution provides the optimal strategies of the players. Additionally, we prove the existence and uniqueness theorem for the deduced system of ODEs with unilateral constraints.
A multi-player competitive Dynkin stopping game is constructed. Each player can either exit the game for a fixed payoff, determined a priori, or stay and receive an adjusted payoff depending on the decision of other players. The single period case is shown to be "weakly unilaterally competitive". We present an explicit construction of the unique value at which Nash and optimal equilibria are attained. Multiple period generalisations are explored. The game has interpretations in economic and financial contexts, for example, as a consumption model with bounded resources. It also serves as a star…
We consider a zero-sum stochastic game for continuous-time Markov chain with countable state space and unbounded transition and pay-off rates. The additional feature of the game is that the controllers together with taking actions are also allowed to stop the process. Under suitable hypothesis we show that the game has a value and it is the unique solution of certain dynamic programming inequalities with bilateral constraints. In the process we also prescribe a saddle point equilibrium.
This paper analyses two-player nonzero-sum games of optimal stopping on a class of linear regular diffusions with not non-singular boundary behaviour (in the sense of Itô and McKean (1974), p.\ 108). We provide sufficient conditions under which Nash equilibria are realised by each player stopping the diffusion at one of the two boundary points of an interval. The boundaries of this interval solve a system of algebraic equations. We also provide conditions sufficient for the uniqueness of the equilibrium in this class.
In this paper we study the optimization problem of an economic agent who chooses a job and the time of retirement as well as consumption and portfolio of assets. The agent is constrained in the ability to borrow against future income. We transform the problem into a dual two-person zero-sum game, which involves a controller, who is a minimizer and chooses a non-increasing process, and a stopper, who is a maximizer and chooses a stopping time. We derive the Hamilton-Jacobi- Bellman quasi-variational inequality(HJBQV) of a max-min type arising from the game. We provide a solution to the HJBQV an…
We provide an algorithm to find the value and an optimal strategy of the solitaire variant of the Ten Thousand dice game in the framework of Markov Control Processes. Once an optimal critical threshold is found, the set of non-stopping states of the game becomes finite, and the solution is found by a backwards algorithm that gives the values for each one of these states of the game. The algorithm is finite and exact.The idea to find the critical threshold comes from the continuous pasting condition used in optimal stopping problems for continuous-time processes with jumps.
Many decision problems in economics, information technology, and industry can be transformed to an optimal stopping of adapted random vectors with some utility function over the set of Markov times with respect to filtration build by the decision maker's knowledge. The optimal stopping problem formulation is to find a stopping time which maximizes the expected value of the accepted (stopped) random vector's utility. There are natural extensions of optimal stopping problem to stopping the games-the problem of stopping random vectors by two or more decision makers. Various approaches dependent o…
A Dynkin game is a zero-sum, stochastic stopping game between two players where either player can stop the game at any time for an observable payoff. Typically the payoff process of the max-player is assumed to be smaller than the payoff process of the min-player, while the payoff process for simultaneous stopping is in between the two. In this paper, we study general Dynkin games whose payoff processes are in arbitrary positions. In both discrete and continuous time settings, we provide necessary and sufficient conditions for the existence of pure strategy Nash equilibria and epsilon-optimal …
We formulate a stochastic game of mean field type where the agents solve optimal stopping problems and interact through the proportion of players that have already stopped. Working with a continuum of agents, typical equilibria become functions of the common noise that all agents are exposed to, whereas idiosyncratic randomness can be eliminated by an Exact Law of Large Numbers. Under a structural monotonicity assumption, we can identify equilibria with solutions of a simple equation involving the distribution function of the idiosyncratic noise. Solvable examples allow us to gain insight into…
We prove the dynamic programming principe for uniformly nondegenerate stochastic differential games in the framework of time-homogeneous diffusion processes considered up to the first exit time from a domain. The zeroth-order "coefficient" and the "free" term are only assumed to be measurable. In contrast with previous results established for constant stopping times we allow arbitrary stopping times and randomized ones as well. The main assumption, which will be removed in a subsequent article, is that there exists a sufficiently regular solution of the Isaacs equation.
We study the existence of optimal actions in a zero-sum game $\inf_τ\sup_PE^P[X_τ]$ between a stopper and a controller choosing a probability measure. This includes the optimal stopping problem $\inf_τ\mathcal{E}(X_τ)$ for a class of sublinear expectations $\mathcal{E}(\cdot)$ such as the $G$-expectation. We show that the game has a value. Moreover, exploiting the theory of sublinear expectations, we define a nonlinear Snell envelope $Y$ and prove that the first hitting time $\inf\{t:Y_t=X_t\}$ is an optimal stopping time. The existence of a saddle point is shown under a compactness condition.…
On a filtered probability space $(Ω,\mathcal{F},P,\mathbb{F}=(\mathcal{F}_t)_{t=0,\dotso,T})$, we consider stopper-stopper games $\overline V:=\inf_{\Rho\in\bT^{ii}}\sup_{τ\in\T}\E[U(\Rho(τ),τ)]$ and $\underline V:=\sup_{\Tau\in\bT^i}\inf_{ρ\in\T}\E[U(\Rho(τ),τ)]$ in discrete time, where $U(s,t)$ is $\mathcal{F}_{s\vee t}$-measurable instead of $\mathcal{F}_{s\wedge t}$-measurable as is often assumed in the literature, $\T$ is the set of stopping times, and $\bT^i$ and $\bT^{ii}$ are sets of mappings from $\T$ to $\T$ satisfying certain non-anticipativity conditions. We convert the problems in…
Under non-exponential discounting, we develop a dynamic theory for stopping problems in continuous time. Our framework covers discount functions that induce decreasing impatience. Due to the inherent time inconsistency, we look for equilibrium stopping policies, formulated as fixed points of an operator. Under appropriate conditions, fixed-point iterations converge to equilibrium stopping policies. This iterative approach corresponds to the hierarchy of strategic reasoning in Game Theory, and provides "agent-specific" results: it assigns one specific equilibrium stopping policy to each agent a…
We prove the dynamic programming principe for uniformly nondegenerate stochastic differential games in the framework of time-homogeneous diffusion processes considered up to the first exit time from a domain. In contrast with previous results established for constant stopping times we allow arbitrary stopping times and randomized ones as well. There is no assumption about solvability of the the Isaacs equation in any sense (classical or viscosity). The zeroth-order "coefficient" and the "free" term are only assumed to be measurable in the space variable. We also prove that value functions are …
We prove the dynamic programming principle (DPP) in a class of problems where an agent controls a $d$-dimensional diffusive dynamics via both classical and singular controls and, moreover, is able to terminate the optimisation at a time of her choosing, prior to a given maturity. The time-horizon of the problem is random and it is the smallest between a fixed terminal time and the first exit time of the state dynamics from a Borel set. We consider both the cases in which the total available fuel for the singular control is either bounded or unbounded. We build upon existing proofs of DPP and e…
We develop a model for the industry dynamics in the electricity market, based on mean-field games of optimal stopping. In our model, there are two types of agents: the renewable producers and the conventional producers. The renewable producers choose the optimal moment to build new renewable plants, and the conventional producers choose the optimal moment to exit the market. The agents interact through the market price, determined by matching the aggregate supply of the two types of producers with an exogenous demand function. Using a relaxed formulation of optimal stopping mean-field games, w…
We use martingale and stochastic analysis techniques to study a continuous-time optimal stopping problem, in which the decision maker uses a dynamic convex risk measure to evaluate future rewards. We also find a saddle point for an equivalent zero-sum game of control and stopping, between an agent (the "stopper") who chooses the termination time of the game, and an agent (the "controller", or "nature") who selects the probability measure.
Dynamic games arise when multiple agents with differing objectives choose control inputs to a dynamic system. Dynamic games model a wide variety of applications in economics, defense, and energy systems. However, compared to single-agent control problems, the computational methods for dynamic games are relatively limited. As in the single-agent case, only very specialized dynamic games can be solved exactly, and so approximation algorithms are required. This paper extends the differential dynamic programming algorithm from single-agent control to the case of non-zero sum full-information dynam…
This paper introduces a new class of Dynkin games, where the two players are allowed to make their stopping decisions at a sequence of exogenous Poisson arrival times. The value function and the associated optimal stopping strategy are characterized by the solution of a backward stochastic differential equation. The paper further applies the model to study the optimal conversion and calling strategies of convertible bonds, and their asymptotics when the Poisson intensity goes to infinity.
We consider a stochastic game of control and stopping specified in terms of a process $X_t=-θΛ_t+W_t$, representing the holdings of Player 1, where $W$ is a Brownian motion, $θ$ is a Bernoulli random variable indicating whether Player 2 is active or not, and $Λ$ is a non-decreasing process representing the accumulated "theft" or "fraud" performed by Player 2 (if active) against Player 1. Player 1 cannot observe $θ$ or $Λ$ directly, but can merely observe the path of the process $X$ and may choose a stopping rule $τ$ to deactivate Player 2 at a cost $M$. Player 1 thus does not know if she is th…
We consider fractional linear programming production games for the single-objective and multiobjective cases. We use the method of Chakraborty and Gupta (2002) in order to transform the fractional linear programming problems into linear programming problems. A cooperative game is attached and we prove the non-emptiness of the core by using the duality theory from the linear programming. In the multiobjective case, we give a characterization of the Stable outcome of the associate cooperative game, which is balanced. We also consider the cooperative game associated to an exchange economy with a …
