We study a mean field optimal control problem with general non-Markovian dynamics, including both common noise and jumps. We show that its minimizers are Nash equilibria of an associated mean field game of controls. These types of games are necessarily potential, and the Nash equilibria derived as the minimizers of the control problem are closely connected to McKean-Vlasov equations of Langevin type. To illustrate the general theory, we present several examples, including a mean field game of controls with interactions through a price variable, and mean field Cucker-Smale Flocking and Kuramoto models. We also establish the invariance property of the value function, a key ingredient used in our proofs.
We consider zero sum stochastic games. For every discount factor $λ$, a time normalization allows to represent the game as being played on the interval [0, 1]. We introduce the trajectories of cumulated expected payoff and of cumulated occupation measure up to time t $\in$ [0, 1], under $ε$-optimal strategies. A limit optimal trajectory is defined as an accumulation point as the discount factor tends to 0. We study existence, uniqueness and characterization of these limit optimal trajectories for absorbing games.
The properties of value functions of time inhomogeneous optimal stopping problem and zero-sum game (Dynkin game) are studied through time dependent Dirichlet form. Under the absolute continuity condition on the transition function of the underlying diffusion process and some other assumptions, the refined solutions without exceptional starting points are proved to exist, and the value functions of the optimal stopping and zero-sum game, which are finely and cofinely continuous, are characterized as the solutions of some variational inequalities, respectively.
In this paper we establish a new connection between a class of 2-player nonzero-sum games of optimal stopping and certain $2$-player nonzero-sum games of singular control. We show that whenever a Nash equilibrium in the game of stopping is attained by hitting times at two separate boundaries, then such boundaries also trigger a Nash equilibrium in the game of singular control. Moreover a differential link between the players' value functions holds across the two games.
This paper studies singular mean field control problems and singular mean field stochastic differential games. Both sufficient and necessary conditions for the optimal controls and for the Nash equilibrium are obtained. Under some assumptions the optimality conditions for singular mean-field control are reduced to a reflected Skorohod problem, whose solution is proved to exist uniquely. Applications are given to optimal harvesting of stochastic mean-field systems, optimal irreversible investments under uncertainty and to mean-field singular investment games. In particular, a simple singular me…
Using the duality techniques introduced by De Meyer (1996a, 1996b), De Meyer and Marino (2005) provided optimal strategies for both players in finitely repeated games with incomplete information on two sides, in the independent case. In this note, we extend both the duality techniques and the construction of optimal strategies to the case of general type dependence.
We develop the linear programming approach to mean-field games in a general setting. This relaxed control approach allows to prove existence results under weak assumptions, and lends itself well to numerical implementation. We consider mean-field game problems where the representative agent chooses both the optimal control and the optimal time to exit the game, where the instantaneous reward function and the coefficients of the state process may depend on the distribution of the other agents. Furthermore, we establish the equivalence between mean-field games equilibria obtained by the linear p…
We show that the value function of an optimal stopping game driven by a one-dimensional diffusion can be characterised using a modification of the Legendre transformation if and only if the optimal stopping game exhibits a Nash equilibrium (i.e. a saddle point of the optimal stopping game exists). This result is an analytical complement to the results in Peskir, G. (2012) A Duality Principle for the Legendre Transform. Journal of Convex Analysis, 19(3), 609-630 where the `duality' between a concave-biconjugate which is modified to remain below an upper barrier and a convex-biconjugate which is…
We develop the fictitious play algorithm in the context of the linear programming approach for mean field games of optimal stopping and mean field games with regular control and absorption. This algorithm allows to approximate the mean field game population dynamics without computing the value function by solving linear programming problems associated with the distributions of the players still in the game and their stopping times/controls. We show the convergence of the algorithm using the topology of convergence in measure in the space of subprobability measures, which is needed to deal with…
We propose a new evolutionary dynamics for population games with a discrete strategy set, inspired by the theory of optimal transport and Mean field games. The dynamics can be described as a Fokker-Planck equation on a discrete strategy set. The derived dynamics is the gradient flow of a free energy and the transition density equation of a Markov process. Such process provides models for the behavior of the individual players in population, which is myopic, greedy and irrational. The stability of the dynamics is governed by optimal transport metric, entropy and Fisher information.
This paper establishes that $N$-player stochastic games with singular controls, either of bounded velocity or of finite variation, can both be approximated by mean field games (MFGs) with singular controls of bounded velocity. More specifically, it shows i) the optimal control to an MFG with singular controls of a bounded velocity $θ$ is shown to be an $ε_N$-NE to an $N$-player game with singular controls of the bounded velocity, with $ε_N = O(\frac{1}{\sqrt{N}})$, and (ii) the optimal control to this MFG is an $(ε_N + ε_θ)$-NE to an $N$-player game with singular controls of finite variation, …
We consider an optimal partition of resources (e.g. consumers) between several agents (e.g. experts), given utility functions ("wisdoms") for the agents and their capacities. This problem is a variant of optimal transport (Monge-Kantorovich) between two measure spaces where one of the measures is discrete (capacities) and the costs of transport are the wisdoms of the agents. We concentrate on the individual value for each agent under optimal partition and show that, counter-intuitively, this value may decrease if the agent's wisdom is increased. Sufficient and necessary conditions for incremen…
This paper establishes the existence of equilibria result of a class of mean field games with singular controls. The interaction takes place through both states and controls. A relaxed solution approach is used. To circumvent the tightness issue, we prove the existence of equilibria by first considering the corresponding mean field games with continuous controls instead of singular controls and then taking approximation.
Continuous and discrete game-theoretic models of dispersed material drying process are formalized and studied in the paper. The existence of optimal drying strategies is shown through application of results from the theory of differential games and dynamic programming. These optimal strategies can be found numerically.
This paper analyses a stochastic differential game of control and stopping in which one of the players modifies a diffusion process using impulse controls, an adversary then chooses a stopping time to end the game. The paper firstly establishes the regularity and boundedness of the upper and lower value functions from which an appropriate variant of the dynamic programming principle (DPP) is derived. It is then proven that the upper and lower value functions coincide so that the game admits a value and that the value of the game is a unique viscosity solution to a HJBI equation described by a …
An interrelationship between Game Theory and Control Theory is seeked. In this respect two aspects of this relationship are brought up. To establish the direct relationship Control Based Games and to establish the inverse relationship Game Based Control are discussed. In the attempt to establish the direct relationship Control Based Boolean Networks are discussed with the novel technique of application of Semi Tensor Product of matrices in Differential Calculus. For the inverse relationship H Infinity robust optimal control has been discussed with the help of Dynamic Programming and Pontryagin…
We provide an algorithm to find the value and an optimal strategy of the solitaire variant of the Ten Thousand dice game in the framework of Markov Control Processes. Once an optimal critical threshold is found, the set of non-stopping states of the game becomes finite, and the solution is found by a backwards algorithm that gives the values for each one of these states of the game. The algorithm is finite and exact.The idea to find the critical threshold comes from the continuous pasting condition used in optimal stopping problems for continuous-time processes with jumps.
We study the optimal use of information in Markov games with incomplete information on one side and two states. We provide a finite-stage algorithm for calculating the limit value as the gap between stages goes to 0, and an optimal strategy for the informed player in the limiting game in continuous time. This limiting strategy induces an-optimal strategy for the informed player, provided the gap between stages is small. Our results demonstrate when the informed player should use his information and how.
The extension of a conflict control problem with infinite horizon is constructed. This extension is the projective limit of restricted games. Relations between "sensitivity to target set" and the existence of the optimal control are studied. Special attention is paid to the pursuit-evasion game with "joint control/relaxed control".
This paper establishes the existence of relaxed solutions to mean field games (MFGs for short) with singular controls. We also prove approximations of solutions results for a particular class of MFGs with singular controls by solutions, respectively control rules, for MFGs with purely regular controls. Our existence and approximation results strongly hinge on the use of the Skorokhod $M_1$ topology on the space of càdlàg functions.
This work studies Nash equilibrium seeking for a class of stochastic aggregative games, where each player has an expectation-valued objective function depending on its local strategy and the aggregate of all players' strategies. We propose a distributed algorithm with operator extrapolation, in which each player maintains an estimate of this aggregate by exchanging this information with its neighbors over a time-varying network, and updates its decision through the mirror descent method. An operator extrapolation at the search direction is applied such that the two step historical gradient sam…
Motivated by the self-pursuit of controlled objects, we consider the exact controllability of a linear mean-field type game-based control system (MF-GBCS, for short) generated by a linear-quadratic (LQ, for short) Nash game. A Gram-type criterion for the general timevarying coefficients case and a Kalman-type criterion for the special time-invariant coefficients case are obtained. At the same time, the equivalence between the exact controllability of this MF-GBCS and the exact observability of a dual system is established. Moreover, an admissible control that can steer the state from any initi…
In the nonzero-sum setting, we establish a connection between Nash equilibria in games of optimal stopping (Dynkin games) and generalised Nash equilibrium problems (GNEP). In the Dynkin game this reveals novel equilibria of threshold type and of more complex types, and leads to novel uniqueness and stability results.
In this paper we study continuous-time two-player zero-sum optimal switching games on a finite horizon. Using the theory of doubly reflected BSDEs with interconnected barriers, we show that this game has a value and an equilibrium in the players' switching controls.
We establish existence of nearly-optimal controls, conditions for existence of an optimal control and a saddle-point for respectively a control problem and zero-sum differential game associated with payoff functionals of mean-field type, under dynamics driven by weak solutions of stochastic differential equations of mean-field type.
We consider the mean-field game where each agent determines the optimal time to exit the game by solving an optimal stopping problem with reward function depending on the density of the state processes of agents still present in the game. We place ourselves in the framework of relaxed optimal stopping, which amounts to looking for the optimal occupation measure of the stopper rather than the optimal stopping time. This framework allows us to prove the existence of the relaxed Nash equilibrium and the uniqueness of the associated value of the representative agent under mild assumptions. Further…
This work considers two-player zero-sum semi-Markov games with incomplete information on one side and perfect observation. At the beginning, the system selects a game type according to a given probability distribution and informs to Player 1 only. After each stage, the actions chosen are observed by both players before proceeding to the next stage. Firstly, we show the existence of the value function under the expected discount criterion and the optimality equation. Secondly, the existence and iterative algorithm of the optimal policy for Player 1 are introduced through the optimality equation…
The article provides a solution algorithm for the linear programming problem (LPP) with the latter being presented as an antagonistic matrix game so the game's further solution is based on the iterative method. The algorithm is presented as a computer program. Having applied necessary accuracy, the author has researched the solution assessment convergence rate in relation to the actual value. Program implementation demonstrates high rate of the LPP solution receipt, with the acceptable accuracy being fractions or unities. It allows using the algorithm in the integrated systems for the purpose …
This paper uses recent results on continuous-time finite-horizon optimal switching problems with negative switching costs to prove the existence of a saddle point in an optimal stopping (Dynkin) game. Sufficient conditions for the game's value to be continuous with respect to the time horizon are obtained using recent results on norm estimates for doubly reflected backward stochastic differential equations. This theory is then demonstrated numerically for the special cases of cancellable call and put options in a Black-Scholes market.
In this paper we study the problem of approximating the general solution to an optimal control problem whose dynamics arise from a $2\times 2$ skew-symmetric evolutionary game with arbitrary initial condition. Our approach uses a Fourier approximation method and generalizes prior work in the use of orthogonal function approximation for optimal control. At the same time we cast the fitting problem in the context of a non-standard feedforward neural network and derive the back-propagation operator in this context. An example of the efficacy of this approach is provided and generalizations are di…
Each of two players, by turns, rolls a dice several times accumulating the successive scores until he decides to stop, or he rolls an ace. When stopping, the accumulated turn score is added to the player account and the dice is given to his opponent. If he rolls an ace, the dice is given to the opponent without adding any point. In this paper we formulate this game in the framework of competitive Markov decision processes (also known as stochastic games), show that the game has a value, provide an algorithm to compute the optimal minimax strategy, and present results of this algorithm in three…
We consider $n$ risk-averse agents who compete for liquidity in an Almgren--Chriss market impact model. Mathematically, this situation can be described by a Nash equilibrium for a certain linear-quadratic differential game with state constraints. The state constraints enter the problem as terminal boundary conditions for finite and infinite time horizons. We prove existence and uniqueness of Nash equilibria and give closed-form solutions in some special cases. We also analyze qualitative properties of the equilibrium strategies and provide corresponding financial interpretations.
We consider a mean field game (MFG) of optimal portfolio liquidation under asymmetric information. We prove that the solution to the MFG can be characterized in terms of a FBSDE with possibly singular terminal condition on the backward component or, equivalently, in terms of a FBSDE with finite terminal value, yet singular driver. Extending the method of continuation to linear-quadratic FBSDE with singular driver we prove that the MFG has a unique solution. Our existence and uniqueness result allows to prove that the MFG with possibly singular terminal condition can be approximated by a sequen…
We consider a general class of nonzero-sum $N$-player stochastic games with impulse controls, where players control the underlying dynamics with discrete interventions. We adopt a verification approach and provide sufficient conditions for the Nash equilibria (NEs) of the game. We then consider the limit situation of $N \to \infty$, that is, a suitable mean-field game (MFG) with impulse controls. We show that under appropriate technical conditions, the existence of unique NE solution to the MFG, which is an $ε$-NE approximation to the $N$-player game, with $ε=O\left(\frac{1}{\sqrt{N}}\right)$.…
The pricing, hedging, optimal exercise and optimal cancellation of game or Israeli options are considered in a multi-currency model with proportional transaction costs. Efficient constructions for optimal hedging, cancellation and exercise strategies are presented, together with numerical examples, as well as probabilistic dual representations for the bid and ask price of a game option.
This work considers stochastic differential games with a large number of players, whose costs and dynamics interact through the empirical distribution of both their states and their controls. We develop a new framework to prove convergence of finite-player games to the asymptotic mean field game. Our approach is based on the concept of propagation of chaos for forward and backward weakly interacting particles which we investigate by stochastic analysis methods, and which appear to be of independent interest. These propagation of chaos arguments allow to derive moment and concentration bounds f…
A fundamental open problem in monotone game theory is the computation of a specific generalized Nash equilibrium (GNE) among all the available ones, e.g. the optimal equilibrium with respect to a system-level objective. The existing GNE seeking algorithms have in fact convergence guarantees toward an arbitrary, possibly inefficient, equilibrium. In this paper, we solve this open problem by leveraging results from fixed-point selection theory and in turn derive distributed algorithms for the computation of an optimal GNE in monotone games. We then extend the technical results to the time-varyin…
We consider static finite-player network games and their continuum analogs, graphon games. Existence and uniqueness results are provided, as well as convergence of the finite-player network game optimal strategy profiles to their analogs for the graphon games. We also show that equilibrium strategy profiles of a graphon game provide approximate Nash equilibria for the finite-player games. Connections with mean field games and central planner optimization problems are discussed. Motivating applications are presented and explicit computations of their Nash equilibria and social optimal strategie…
This paper aims to design a distributed coordination algorithm for solving a multi-agent decision problem with a hierarchical structure. The primary goal is to search the Nash equilibrium of a noncooperative game such that each player has no incentive to deviate from the equilibrium under its private objective. Meanwhile, the agents can coordinate to optimize the social cost within the set of Nash equilibria of the underlying game. Such an optimal Nash equilibrium problem can be modeled as a distributed optimization problem with variational inequality constraints. We consider the scenario wher…
In this article, we study the repeated routing game problem on a parallel network with affine latency functions on each edge. We cast the game setup in a LQR control theoretic framework, leveraging the Rosenthal potential formulation. We use control techniques to analyze the convergence of the game dynamics with specific cases that lend themselves to optimal control. We design proper dynamics parameters so that the conservation of flow is guaranteed. We provide an algorithmic solution for the general optimal control setup using a multiparametric quadratic programming approach (explicit MPC). F…
Two-player pursuit-evasion differential game and time optimal zero control problem in $\ell^2$ are considered. Optimal control for the corresponding zero control problem is found. A strategy for the pursuer that guarantees the solution for the pursuit problem is constructed.
We establish existence of Markov chains of mean-field type with unbounded jump intensities by means of a fixed point argument using the Total Variation distance. We further show existence of nearly-optimal controls and, using a Markov chain backward SDE approach, we suggest conditions for existence of an optimal control and a saddle-point for respectively a control problem and a zero-sum differential game associated with payoff functionals of mean-field type, under dynamics driven by such Markov chains of mean-field type.
