In the context of simple finite-state discrete time systems, we introduce a generalization of mean field game solution, called correlated solution, which can be seen as the mean field game analogue of a correlated equilibrium. Our notion of solution is justified in two ways: We prove that correlated solutions arise as limits of exchangeable correlated equilibria in restricted (Markov open-loop) strategies for the underlying $N$-player games, and we show how to construct approximate $N$-player correlated equilibria starting from a correlated solution to the mean field game.
We introduce a notion of subgames for stochastic timing games and the related notion of subgame-perfect equilibrium in possibly mixed strategies. While a good notion of subgame-perfect equilibrium for continuous-time games is not available in general, we argue that our model is the appropriate version for timing games. We show that the notion coincides with the usual one for discrete-time games. Many timing games in continuous time have only equilibria in mixed strategies -- in particular preemption games, which often occur in the strategic real option literature. We provide a sound foundation…
In a discrete space and time framework, we study the mean field game limit for a class of symmetric $N$-player games based on the notion of correlated equilibrium. We give a definition of correlated solution that allows to construct approximate $N$-player correlated equilibria that are robust with respect to progressive deviations. We illustrate our definition by way of an example with explicit solutions.
This paper continues the study of the mean field game (MFG) convergence problem: In what sense do the Nash equilibria of $n$-player stochastic differential games converge to the mean field game as $n\rightarrow\infty$? Previous work on this problem took two forms. First, when the $n$-player equilibria are open-loop, compactness arguments permit a characterization of all limit points of $n$-player equilibria as weak MFG equilibria, which contain additional randomness compared to the standard (strong) equilibrium concept. On the other hand, when the $n$-player equilibria are closed-loop, the con…
We study a class of dynamic decision problems of mean field type with time inconsistent cost functionals, and derive a stochastic maximum principle to characterize subgame perfect Nash equilibrium points. Subsequently, this approach is extended to a mean field game to construct decentralized strategies and obtain an estimate of their performance.
Mean field games are studied in the framework of controlled martingale problems, and general existence theorems are proven in which the equilibrium control is Markovian. The framework is flexible enough to include degenerate volatility, which may depend on both the control and the mean field. The objectives need not be strictly convex, and the mean field interactions considered are nonlocal and Wasserstein-continuous. When the volatility is nondegenerate, continuity assumptions may be weakened considerably. The proofs first use relaxed controls to establish existence. Then, using a convexity a…
We study the convergence of Nash equilibria in a game of optimal stopping. If the associated mean field game has a unique equilibrium, any sequence of $n$-player equilibria converges to it as $n\to\infty$. However, both the finite and infinite player versions of the game often admit multiple equilibria. We show that mean field equilibria satisfying a transversality condition are limit points of $n$-player equilibria, but we also exhibit a remarkable class of mean field equilibria that are not limits, thus questioning their interpretation as "large $n$" equilibria.
In this paper, we consider constrained discounted stochastic games with a countably generated state space and norm continuous transition probability having a density function. We prove existence of approximate stationary equilibria and stationary weak correlated equilibria. Our results imply the existence of stationary Nash equilibrium in $ARAT$ stochastic games.
Mean field games allow to describe tractable models of dynamic games with a continuum of players, explicit interaction and heterogeneous states. Thus, these models are of great interest for socio-economic applications. A particular class of these models are games with finite state and action space, for which recently in Neumann (2020a) a semi-explicit representation of all stationary equilibria has been obtained. In this paper we investigate whether these stationary equilibria are stable against model perturbations. We prove that the set of all games with only essential equilibria is residual …
We present a unified framework for characterizing local Nash equilibria in continuous games on either infinite-dimensional or finite-dimensional non-convex strategy spaces. We provide intrinsic necessary and sufficient first- and second-order conditions ensuring strategies constitute local Nash equilibria. We term points satisfying the sufficient conditions differential Nash equilibria. Further, we provide a sufficient condition (non-degeneracy) guaranteeing differential Nash equilibria are isolated and show that such equilibria are structurally stable. We present tutorial examples to illustra…
We study a static game played by a finite number of agents, in which agents are assigned independent and identically distributed random types and each agent minimizes its objective function by choosing from a set of admissible actions that depends on its type. The game is anonymous in the sense that the objective function of each agent depends on the actions of other agents only through the empirical distribution of their type-action pairs. We study the asymptotic behavior of Nash equilibria, as the number of agents tends to infinity, first by deriving laws of large numbers characterizes almos…
We observe that a significant class of Nash equilibrium problems in non-potential games can be associated with monotone inclusion problems. We propose splitting techniques to solve such problems and establish their convergence. Applications to generalized Nash equilibria, zero-sum games, and cyclic proximity problems are demonstrated.
A bimatrix game $(A,B)$ is called a game of rank $k$ if the rank of the matrix $A+B$ is at most $k$. We consider the problem of enumerating the Nash equilibria in (non-degenerate) games of rank 1. In particular, we show that even for games of rank 1 not all equilibria can be reached by a Lemke-Howson path and present a parametric simplex-type algorithm for enumerating all Nash equilibria of a non-degenerate game of rank 1.
We consider mean field games with discrete state spaces (called discrete mean field games in the following) and we analyze these games in continuous and discrete time, over finite as well as infinite time horizons. We prove the existence of a mean field equilibrium assuming continuity of the cost and of the drift. These conditions are more general than the existing papers studying finite state space mean field games. Besides, we also study the convergence of the equilibria of N -player games to mean field equilibria in our four settings. On the one hand, we define a class of strategies in whic…
Any nonempty, compact, semi-algebraic set in [0, 1] n is the projection of the set of mixed equilibria of a finite game with 2 actions per player on its first n coordinates. A similar result follows for sets of equilibrium payoffs. The proofs are constructive and elementary.
In this paper we consider continuity of the set of Nash equilibria and approximate Nash equilibria for parameterized games. For parameterized games with unique Nash equilibria, the continuity of this equilibrium mapping is well-known. However, when the equilibria need not be unique, there may exist discontinuities in the equilibrium mapping. The focus of this work is to summarize continuity properties for parameterized Nash equilibria and prove continuity via the approximate Nash game with uniformly continuous objective functions over potentially non-compact strategy spaces.
The existence of stationary Markov perfect equilibria in stochastic games is shown under a general condition called "(decomposable) coarser transition kernels". This result covers various earlier existence results on correlated equilibria, noisy stochastic games, stochastic games with finite actions and state-independent transitions, and stochastic games with mixtures of constant transition kernels as special cases. A remarkably simple proof is provided via establishing a new connection between stochastic games and conditional expectations of correspondences. New applications of stochastic gam…
We study games with finitely many participants, each having finitely many choices. We consider the following categories of participants: (I) populations: sets of nonatomic agents, (II) atomic splittable players, (III) atomic non splittable players. We recall and compare the basic properties, expressed through variational inequalities, concerning equilibria, potential games and dissipative games, as well as evolutionary dynamics. Then we consider composite games where the three categories of participants are present, a typical example being congestion games, and extend the previous properties o…
We consider a class of N-player stochastic games of multi-dimensional singular control, in which each player faces a minimization problem of monotone-follower type with submodular costs. We call these games "monotone-follower games". In a not necessarily Markovian setting, we establish the existence of Nash equilibria. Moreover, we introduce a sequence of approximating games by restricting, for each natural number n, the players' admissible strategies to the set of Lipschitz processes with Lipschitz constant bounded by n. We prove that, for each n, there exists a Nash equilibrium of the approx…
We construct an approximate public-signal correlated equilibrium for a nonzero-sum differential game in the class of stochastic strategies with memory. The construction is based on a solution of an auxiliary nonzero-sum continuous-time stochastic game. This class of games includes stochastic differential games and continuous-time Markov games. Moreover, we study the limit of approximate equilibrium outcomes in the case when the auxiliary stochastic games tend to the original deterministic one. We show that it lies in the convex hull of the set of equilibrium values provided by deterministic pu…
We prove that every repeated game with countably many players, finite action sets, and tail-measurable payoffs admits an $ε$-equilibrium, for every $ε> 0$.
We consider multi-agent decision making, where each agent optimizes its cost function subject to constraints. Agents' actions belong to a compact convex Euclidean space and the agents' cost functions are coupled. We propose a distributed payoff-based algorithm to learn Nash equilibria in the game between agents. Each agent uses only information about its current cost value to compute its next action. We prove convergence of the proposed algorithm to a Nash equilibrium in the game leveraging established results on stochastic processes. The performance of the algorithm is analyzed with a numeric…
Mean field games (MFG) are dynamic games with infinitely many infinitesimal agents. In this context, we study the efficiency of Nash MFG equilibria: Namely, we compare the social cost of a MFG equilibrium with the minimal cost a global planner can achieve. We find a structure condition on the game under which there exists efficient MFG equilibria and, in case this condition is not fulfilled, quantify how inefficient MFG equilibria are.
We consider multi-agent decision making where each agent optimizes its convex cost function subject to individual and coupling constraints. The constraint sets are compact convex subsets of a Euclidean space. To learn Nash equilibria, we propose a novel distributed payoff-based algorithm, such that each agent uses information only about its cost function values and the constraint function values with their associated dual multiplier. We prove convergence of this algorithm to a Nash equilibrium, under the assumption that the game admits a strictly convex potential function. In the absence of co…
We study uniqueness of Nash equilibria in atomic splittable congestion games and derive a uniqueness result based on polymatroid theory: when the strategy space of every player is a bidirectional flow polymatroid, then equilibria are unique. Bidirectional flow polymatroids are introduced as a subclass of polymatroids possessing certain exchange properties. We show that important cases such as base orderable matroids can be recovered as a special case of bidirectional flow polymatroids. On the other hand we show that matroidal set systems are in some sense necessary to guarantee uniqueness of e…
If a game has a unique Nash equilibrium, then this equilibrium is arguably the solution of the game from the refinement's literature point of view. However, it might be that for almost all initial conditions, all strategies in the support of this equilibrium are eliminated by the replicator dynamics and the best-reply dynamics.
This paper shows the existence of $\mathcal{O}(\frac{1}{n^γ})$-Nash equilibria in $n$-player noncooperative sum-aggregative games in which the players' cost functions, depending only on their own action and the average of all players' actions, are lower semicontinuous in the former while $γ$-Hölder continuous in the latter. Neither the action sets nor the cost functions need to be convex. For an important class of sum-aggregative games, which includes congestion games with $γ$ equal to 1, a gradient-proximal algorithm is used to construct $\mathcal{O}(\frac{1}{n})$-Nash equilibria with at most…
We consider the basic problem of approximating Nash equilibria in noncooperative games. For monotone games, we design continuous time flows which converge in an averaged sense to Nash equilibria. We also study mean field equilibria, which arise in the large player limit of symmetric noncooperative games. In this setting, we will additionally show that the approximation of mean field equilibria is possible under a suitable monotonicity hypothesis.
University of California, Berkeley（加州大学伯克利分校）；Department of Electrical Engineering and Computer Science（电气工程与计算机科学系）；Division of Computer Science（计算机科学部）；Department of Statistics（统计学系）
We propose local symplectic surgery, a two-timescale procedure for finding local Nash equilibria in two-player zero-sum games. We first show that previous gradient-based algorithms cannot guarantee convergence to local Nash equilibria due to the existence of non-Nash stationary points. By taking advantage of the differential structure of the game, we construct an algorithm for which the local Nash equilibria are the only attracting fixed points. We also show that the algorithm exhibits no oscillatory behaviors in neighborhoods of equilibria and show that it has the same per-iteration complexit…
This paper considers dynamic (multi-stage) signaling games involving an encoder and a decoder who have subjective models on the cost functions. We consider both Nash (simultaneous-move) and Stackelberg (leader-follower) equilibria of dynamic signaling games under quadratic criteria. For the multi-stage scalar cheap talk, we show that the final stage equilibrium is always quantized and under further conditions the equilibria for all time stages must be quantized. In contrast, the Stackelberg equilibria are always fully revealing. In the multi-stage signaling game where the transmission of a Gau…
For any two-by-two game $\G$, we define a new two-player game $\G^Q$. The definition is motivated by a vision of players in game $\G$ communicating via quantum technology according to a certain standard protocol originally introduced by Eisert and Wilkins [EW]. In the game $\G^Q$, each players' strategy set consists of the set of all probability distributions on the 3-sphere $S^3$. Nash equilibria in this game can be difficult to compute. Our main theorems classify all possible equilibria in $\G^Q$ for a Zariski-dense set of games $\G$ that we call {\it generic games}. First, we show that up t…
In this paper, we compute $ε$-approximate Nash equilibria in atomic splittable polymatroid congestion games with convex Lipschitz continuous cost functions. The main approach relies on computing a pure Nash equilibrium for an associated integrally-splittable congestion game, where players can only split their demand in integral multiples of a common packet size. It is known that one can compute pure Nash equilibria for integrally-splittable congestion games within a running time that is pseudo-polynomial in the aggregated demand of the players. As the main contribution of this paper, we decide…
Characterization and Computation of Feedback Nash Equilibria in Scalar Discounted N-Player Linear Quadratic Games
IMT School for Advanced Studies Lucca（卢卡高等研究院）；Research Unit DYSCO (Dynamical Systems, Control, and Optimization) at IMT School for Advanced Studies Lucca（卢卡高等研究院动态系统、控制与优化研究单元）
We address the problem of assessing the robustness of the equilibria in uncertain, multi-agent games. Specifically, we focus on generalized Nash equilibrium problems in aggregative form subject to linear coupling constraints affected by uncertainty with a possibly unknown probability distribution. Within a data-driven context, we apply the scenario approach paradigm to provide a-posteriori feasibility certificates for the entire set of generalized Nash equilibria of the game. Then, we show that assessing the violation probability of such set merely requires to enumerate the constraints that ``…
