In [14], Gueant, Lasry and Lions considered the model problem ``What time does meeting start?'' as a prototype for a general class of optimization problems with a continuum of players, called Mean Field Games problems. In this paper we consider a similar model, but with the dynamics of the agents defined on a network. We discuss appropriate transition conditions at the vertices which give a well posed problem and we present some numerical results.
Mean Field Game systems describe equilibrium configurations in differential games with infinitely many infinitesimal interacting agents. We introduce a learning procedure (similar to the Fictitious Play) for these games and show its convergence when the Mean Field Game is potential.
We introduce the notion of stable solution in mean field game theory: they are locally isolated solutions of the mean field game system. We prove that such solutions exist in potential mean field games and are local attractors for learning procedures.
We introduce a model of anonymous games with the player dependent action sets. We propose several learning procedures based on the well-known Fictitious Play and Online Mirror Descent and prove their convergence to equilibrium under the classical monotonicity condition. Typical examples are first-order mean field games.
For a class of finite horizon first order mean field games and associated N-player games, we give a simple proof of convergence of symmetric N-player Nash equilibria in distributed open-loop strategies to solutions of the mean field game in Lagrangian form. Lagrangian solutions are then connected with those determined by the usual mean field game system of two coupled first order PDEs, and convergence of Nash equilibria in distributed Markov strategies is established.
We study a general linear quadratic mean field type control problem and connect it to mean field games of a similar type. The solution is given both in terms of a forward/backward system of stochastic differential equations and by a pair of Riccati equations. In certain cases, the solution to the mean field type control is also the equilibrium strategy for a class of mean field games. We use this fact to study an economic model of production of exhaustible resources. Keywords: mean field type control, mean field games, linear-quadratic, optimal control, riccati equations, exhaustible resource …
In this article we consider finite Mean Field Games (MFGs), i.e. with finite time and finite states. We adopt the framework introduced in Gomes Mohr and Souza in 2010, and study two seemly unexplored subjects. In the first one, we analyze the convergence of the fictitious play learning procedure, inspired by the results in continuous MFGs. In the second one, we consider the relation of some finite MFGs and continuous first order MFGs. Namely, given a continuous first order MFG problem and a sequence of refined space/time grids, we construct a sequence finite MFGs whose solutions admit limits p…
We consider mean field games with discrete state spaces (called discrete mean field games in the following) and we analyze these games in continuous and discrete time, over finite as well as infinite time horizons. We prove the existence of a mean field equilibrium assuming continuity of the cost and of the drift. These conditions are more general than the existing papers studying finite state space mean field games. Besides, we also study the convergence of the equilibria of N -player games to mean field equilibria in our four settings. On the one hand, we define a class of strategies in whic…
We present an example of symmetric ergodic $N$-players differential games, played in memory strategies on the position of the players, for which the limit set, as $N\to +\infty$, of Nash equilibrium payoffs is large, although the game has a single mean field game equilibrium. This example is in sharp contrast with a result by Lacker [23] for finite horizon problems.
The theory of mean field games is a tool to understand noncooperative dynamic stochastic games with a large number of players. Much of the theory has evolved under conditions ensuring uniqueness of the mean field game Nash equilibrium. However, in some situations, typically involving symmetry breaking, non-uniqueness of solutions is an essential feature. To investigate the nature of non-unique solutions, this paper focuses on the technically simple setting where players have one of two states, with continuous time dynamics, and the game is symmetric in the players, and players are restricted t…
This paper establishes the existence of equilibria result of a class of mean field games with singular controls. The interaction takes place through both states and controls. A relaxed solution approach is used. To circumvent the tightness issue, we prove the existence of equilibria by first considering the corresponding mean field games with continuous controls instead of singular controls and then taking approximation.
This paper studies singular mean field control problems and singular mean field stochastic differential games. Both sufficient and necessary conditions for the optimal controls and for the Nash equilibrium are obtained. Under some assumptions the optimality conditions for singular mean-field control are reduced to a reflected Skorohod problem, whose solution is proved to exist uniquely. Applications are given to optimal harvesting of stochastic mean-field systems, optimal irreversible investments under uncertainty and to mean-field singular investment games. In particular, a simple singular me…
The price of anarchy, originally introduced to quantify the inefficiency of selfish behavior in routing games, is extended to mean field games. The price of anarchy is defined as the ratio of a worst case social cost computed for a mean field game equilibrium to the optimal social cost as computed by a central planner. We illustrate properties of such a price of anarchy on linear quadratic extended mean field games, for which explicit computations are possible. Various asymptotic behaviors of the price of anarchy are proved for limiting behaviors of the coefficients in the model and numerics a…
We study the asymptotic organization among many optimizing individuals interacting in a suitable "moderate" way. We justify this limiting game by proving that its solution provides approximate Nash equilibria for large but finite player games. This proof depends upon the derivation of a law of large numbers for the empirical processes in the limit as the number of players tends to infinity. Because it is of independent interest, we prove this result in full detail. We characterize the solutions of the limiting game via a verification argument.
The paper is concerned with the feedback approach to the deterministic mean field type differential games. Previously, it was shown that suboptimal strategies in the mean field type differential game can constructed based on functions of time and probability satisfying the stability condition. This property realizes the dynamic programming principle for the constant control of one player. We present the infinitesimal form of this condition involving analogs of the directional derivatives. In particular, we obtain the characterization of the value function of the deterministic mean field type d…
This paper considers mean field games in a multi-agent Markov decision process (MDP) framework. Each player has a continuum state and binary action. By active control, a player can bring its state to a resetting point. All players are coupled through their cost functions. The structural property of the individual strategies is characterized in terms of threshold policies when the mean field game admits a solution. We further introduce a stationary equation system of the mean field game and analyze uniqueness of its solution under positive externalities.
Mean-Field Games are games with a continuum of players that incorporate the time-dimension through a control-theoretic approach. Recently, simpler approaches relying on the Best Reply Strategy have been proposed. They assume that the agents navigate their strategies towards their goal by taking the direction of steepest descent of their cost function (i.e. the opposite of the utility function). In this paper, we explore the link between Mean-Field Games and the Best Reply Strategy approach. This is done by introducing a Model Predictive Control framework, which consists of setting the Mean-Fie…
We apply an approach based on parametric mean-payoff games to develop bisection and Newton schemes for solving problems of tropical pseudolinear and pseudoquadratic optimisation with general two-sided constraints.
This paper studies the connection between a class of mean-field games and a social welfare optimization problem. We consider a mean-field game in function spaces with a large population of agents, and each agent seeks to minimize an individual cost function. The cost functions of different agents are coupled through a mean-field term that depends on the mean of the population states. We show that although the mean-field game is not a potential game, under some mild condition the $ε$-Nash equilibrium of the mean-field game coincides with the optimal solution to a social welfare optimization pro…
The aim of this paper is to study first order Mean field games subject to a linear controlled dynamics on $\mathbb R^{d}$. For this kind of problems, we define Nash equilibria (called Mean Field Games equilibria), as Borel probability measures on the space of admissible trajectories, and mild solutions as solutions associated with such equilibria. Moreover, we prove the existence and uniqueness of mild solutions and we study their regularity: we prove Hölder regularity of Mean Field Games equilibria and fractional semiconcavity for the value function of the underlying optimal control problem. …
We investigate a mean field game model for the production of exhaustible resources. In this model, firms produce comparable goods, strategically set their production rate in order to maximise profit, and leave the market as soon as they deplete their capacities. We examine the related Mean Field Game system and prove well-posedness for initial measure data by deriving suitable a priori estimates. Then, we show that feedback strategies which are computed from the Mean Field Game system provide $\varepsilon$-Nash equilibria to the corresponding $N$-Player Cournot game, for large values of $N$. T…
This paper establishes that $N$-player stochastic games with singular controls, either of bounded velocity or of finite variation, can both be approximated by mean field games (MFGs) with singular controls of bounded velocity. More specifically, it shows i) the optimal control to an MFG with singular controls of a bounded velocity $θ$ is shown to be an $ε_N$-NE to an $N$-player game with singular controls of the bounded velocity, with $ε_N = O(\frac{1}{\sqrt{N}})$, and (ii) the optimal control to this MFG is an $(ε_N + ε_θ)$-NE to an $N$-player game with singular controls of finite variation, …
Recently developed toy models for the mean-field games of corruption and botnet defence in cyber-security with three or four states of agents are extended to a more general mean-field-game model with $2d$ states, $d\in \N$. In order to tackle new technical difficulties arising from a larger state-space we introduce new asymptotic regimes, namely small discount and small interaction asymptotics. Moreover, the link between stationary and time-dependent solutions is established rigorously leading to a performance of the turnpike theory in a mean-field-game setting.
We develop the fictitious play algorithm in the context of the linear programming approach for mean field games of optimal stopping and mean field games with regular control and absorption. This algorithm allows to approximate the mean field game population dynamics without computing the value function by solving linear programming problems associated with the distributions of the players still in the game and their stopping times/controls. We show the convergence of the algorithm using the topology of convergence in measure in the space of subprobability measures, which is needed to deal with…
We develop the linear programming approach to mean-field games in a general setting. This relaxed control approach allows to prove existence results under weak assumptions, and lends itself well to numerical implementation. We consider mean-field game problems where the representative agent chooses both the optimal control and the optimal time to exit the game, where the instantaneous reward function and the coefficients of the state process may depend on the distribution of the other agents. Furthermore, we establish the equivalence between mean-field games equilibria obtained by the linear p…
We consider N-player and mean field games in continuous time over a finite horizon, where the position of each agent belongs to {-1,1}. If there is uniqueness of mean field game solutions, e.g. under monotonicity assumptions, then the master equation possesses a smooth solution which can be used to prove convergence of the value functions and of the feedback Nash equilibria of the N-player game, as well as a propagation of chaos property for the associated optimal trajectories. We study here an example with anti-monotonous costs, and show that the mean field game has exactly three solutions. W…
A mean field type differential game is a mathematical model of a large system of identical agents under mean-field interaction controlled by two players with opposite purposes. We study the case when the dynamics of each agent is given by ODE and the players can observe the distribution of the agents. We construct suboptimal strategies and prove the existence of the value function.
The paper studies the convergence, as $N$ tends to infinity, of a system of $N$ coupled Hamilton-Jacobi equations (the Nash system) when the coupling between the players becomes increasingly singular. The limit equation is a mean field game system with local coupling.
Existence and uniqueness of a weak solution for first order mean field game systems with local coupling are obtained by variational methods. This solution can be used to devise $ε-$Nash equilibria for deterministic differential games with a finite (but large) number of players. For smooth data, the first component of the weak solution of the MFG system is proved to satisfy (in a viscosity sense) a time-space degenerate elliptic differential equation.
This paper studies the connections between mean-field games and the social welfare optimization problems. We consider a mean field game in functional spaces with a large population of agents, each of which seeks to minimize an individual cost function. The cost functions of different agents are coupled through a mean field term that depends on the mean of the population states. We show that under some mild conditions any $ε$-Nash equilibrium of the mean field game coincides with the optimal solution to a convex social welfare optimization problem. The results are proved based on a general form…
Mean field games allow to describe tractable models of dynamic games with a continuum of players, explicit interaction and heterogeneous states. Thus, these models are of great interest for socio-economic applications. A particular class of these models are games with finite state and action space, for which recently in Neumann (2020a) a semi-explicit representation of all stationary equilibria has been obtained. In this paper we investigate whether these stationary equilibria are stable against model perturbations. We prove that the set of all games with only essential equilibria is residual …
We analyze the behavior of a large number of strategic drivers traveling over an urban traffic network using the mean-field game framework. We assume an incentive mechanism for congestion mitigation under which each driver selecting a particular route is charged a tax penalty that is affine in the logarithm of the number of agents selecting the same route. We show that the mean-field approximation of such a large-population dynamic game leads to the so-called linearly solvable Markov decision process, implying that an open-loop $ε$-Nash equilibrium of the original game can be found simply by s…
This paper proposes and studies a class of discrete-time finite-time-horizon Stackelberg mean-field games, with one leader and an infinite number of identical and indistinguishable followers. In this game, the objective of the leader is to maximize her reward considering the worst-case cost over all possible $ε$-Nash equilibria among followers. A new analytical paradigm is established by showing the equivalence between this Stackelberg mean-field game and a minimax optimization problem. This optimization framework facilitates studying both analytically and numerically the set of Nash equilibri…
We consider deterministic mean field games where the dynamics of a typical agent is non-linear with respect to the state variable and affine with respect to the control variable. Particular instances of the problem considered here are mean field games with control on the acceleration. We focus our attention on the approximation of such mean field games by analogous problems in discrete time and finite state space. For these approximations, we show the existence and, under an additional monotonicity assumption, uniqueness of solutions. In our main result, we establish the convergence of equilib…
We propose and investigate a discrete-time mean field game model involving risk-averse agents. The model under study is a coupled system of dynamic programming equations with a Kolmogorov equation. The agents' risk aversion is modeled by composite risk measures. The existence of a solution to the coupled system is obtained with a fixed point approach. The corresponding feedback control allows to construct an approximate Nash equilibrium for a related dynamic game with finitely many players.
In this article, we consider mean field games between a dominating player and a group of representative agents, each of which acts similarly and also interacts with each other through a mean field term being substantially influenced by the dominating player. We first provide the general theory and discuss the necessary condition for the optimal controls and game condition by adopting adjoint equation approach. We then present a special case in the context of linear-quadratic framework, in which a necessary and sufficient condition can be asserted by stochastic maximum principle; we finally est…
This paper establishes the existence of relaxed solutions to mean field games (MFGs for short) with singular controls. We also prove approximations of solutions results for a particular class of MFGs with singular controls by solutions, respectively control rules, for MFGs with purely regular controls. Our existence and approximation results strongly hinge on the use of the Skorokhod $M_1$ topology on the space of càdlàg functions.
In this paper, we analyze mean-field game modulated by finite states markov chains. We first develop a sufficient stochastic maximum principle for the optimal control of a Markov-modulated stochastic differential equation (SDE) of mean-field type whose coefficients depend on the state of the process, some functional of its law as well as variation of time and sample. As coefficients are perturbed by a Markov chain and thus random, to study such SDEs, we analyze existence and uniqueness of solutions of a class of mean-field type SDEs whose coefficients are random Lipschitz as well as the proper…
For a mean field game model with a major and infinite minor players, we characterize a notion of Nash equilibrium via a system of so-called master equations, namely a system of nonlinear transport equations in the space of measures. Then, for games with a finite number N of minor players and a major player, we prove that the solution of the corresponding Nash system converges to the solution of the system of master equations as N tends to infinity.
We consider a system of mean field games with local coupling in the deterministic limit. Under general structure conditions on the Hamiltonian and coupling, we prove existence and uniqueness of the weak solution, characterizing this solution as the minimizer of some optimal control of Hamilton-Jacobi and continuity equations. We also prove that this solution converges in the long time average to the solution of the associated ergodic problem.
This study addresses primal-dual dynamics for a stochastic programming problem for capacity network design. It is proven that consensus can be achieved on the \textit{here and now} variables which represent the capacity of the network. The main contribution is a heuristic approach which involves the formulation of the problem as a mean-field game. Every agent in the mean-field game has control over its own primal-dual dynamics and seeks consensus with neighboring agents according to a communication topology. We obtain theoretical results concerning the existence of a mean-field equilibrium. Mo…
This paper focuses on linear-quadratic (LQ for short) mean-field games described by forward-backward stochastic differential equations (FBSDEs for short), in which the individual control region is postulated to be convex. The decentralized strategies and consistency condition are represented by a kind of coupled mean-field FBSDEs with projection operators. The well-posedness of consistency condition system is obtained using the monotonicity condition method. The $ε$-Nash equilibrium property is discussed as well.
Mean Field Games with state constraints are differential games with infinitely many agents, each agent facing a constraint on his state. The aim of this paper is to provide a meaning of the PDE system associated with these games, the so-called Mean Field Game system with state constraints. For this, we show a global semiconvavity property of the value function associated with optimal control problems with state constraints.
This note highlights a special class of mean field games in which the coefficients satisfy a convolution-type structural condition. A mean field game of this type with common noise is related to a certain mean field game without common noise by a simple transformation, which permits a tractable construction of a solution of the problem with common noise from a solution of the problem without.
We consider a Mean Field Games model where the dynamics of the agents is given by a controlled Langevin equation and the cost is quadratic. A change of variables, introduced in [9], transforms the Mean Field Games system into a system of two coupled kinetic Fokker-Planck equations. We prove an existence result for the latter system, obtaining consequently existence of a solution for the Mean Field Games system.
This work establishes the equivalence between Mean Field Game and a class of PDE systems closely related to compressible Navier-Stokes equations. The solvability of the PDE system via the existence of the Nash Equilibrium of the Mean Field Game is provided under a set of conditions.
We consider mean field games with ergodic cost in the framework of a general discrete time controlled Markov processes. The state space of the processes is given by a general $σ$-compact Polish space. Under certain conditions, we show the existence of a mean field game equilibrium. We also study the $N$-person game where the players interacts with each other via their empirical measure. We show that the $N$-person game has Nash equilibrium and as $N$ tends to infinity the equilibria converge to a mean field game solution.
The paper is concerned with the deterministic limit of mean field games with the nonlocal coupling. It is assumed that the dynamics of mean field games are given by nonlinear Markov processes. This type of games includes stochastic mean field games as well as mean field games with finite state space. We consider the limiting deterministic mean field game within the framework of minimax approach. The concept of minimax solutions is close to the probabilistic formulation. In this case the Hamilton--Jacobi equation is considered in the minimax/viscosity sense, whereas the flow of probabilities is…
We show that the long time average of solutions of first order mean field game systems in finite horizon is governed by an ergodic system of mean field game type. The well-posedness of this later system and the uniqueness of the ergodic constant rely on weak KAM theory.
A general class of mean field games are considered where the governing dynamics are controlled diffusions in $\mathbb{R}^d$. The optimization criterion is the long time average of a running cost function. Under various sets of hypotheses, we establish the existence of mean field game solutions. We also study the long time behavior of mean field game solutions associated with the finite horizon problem, and under the assumption of geometric ergodicity for the dynamics, we show that these converge to the ergodic mean field game solution as the horizon tends to infinity. Lastly, we study the asso…
In the context of simple finite-state discrete time systems, we introduce a generalization of mean field game solution, called correlated solution, which can be seen as the mean field game analogue of a correlated equilibrium. Our notion of solution is justified in two ways: We prove that correlated solutions arise as limits of exchangeable correlated equilibria in restricted (Markov open-loop) strategies for the underlying $N$-player games, and we show how to construct approximate $N$-player correlated equilibria starting from a correlated solution to the mean field game.
We introduce a mean field model for optimal holding of a representative agent of her peers as a natural expected scaling limit from the corresponding $N-$agent model. The induced mean field dynamics appear naturally in a form which is not covered by standard McKean-Vlasov stochastic differential equations. We study the corresponding mean field game of mutual holding in the absence of common noise. Our first main result provides an explicit equilibrium of this mean field game, defined by a bang--bang control consisting in holding those competitors with positive drift coefficient of their dynami…
