In a discrete space and time framework, we study the mean field game limit for a class of symmetric $N$-player games based on the notion of correlated equilibrium. We give a definition of correlated solution that allows to construct approximate $N$-player correlated equilibria that are robust with respect to progressive deviations. We illustrate our definition by way of an example with explicit solutions.
In the context of simple finite-state discrete time systems, we introduce a generalization of mean field game solution, called correlated solution, which can be seen as the mean field game analogue of a correlated equilibrium. Our notion of solution is justified in two ways: We prove that correlated solutions arise as limits of exchangeable correlated equilibria in restricted (Markov open-loop) strategies for the underlying $N$-player games, and we show how to construct approximate $N$-player correlated equilibria starting from a correlated solution to the mean field game.
This paper deals with an extension of the concept of correlated strategies to Markov stopping games. The Nash equilibrium approach to solving nonzero-sum stopping games may give multiple solutions. An arbitrator can suggest to each player the decision to be applied at each stage based on a joint distribution over the players' decisions. This is a form of equilibrium selection. Examples of correlated equilibria in nonzero-sum games related to the staff selection competition in the case of two departments are given. Utilitarian, egalitarian, republican and libertarian concepts of correlated equi…
In this paper, we consider constrained discounted stochastic games with a countably generated state space and norm continuous transition probability having a density function. We prove existence of approximate stationary equilibria and stationary weak correlated equilibria. Our results imply the existence of stationary Nash equilibrium in $ARAT$ stochastic games.
Optimal Coarse Correlated Equilibria in Mean Field Games: Linear Programming and No-Regret Learning
Department of Mathematics “Federigo Enriques”, University of Milan, Via Saldini 50, 20133, Milan, Italy.；Center for Mathematical Economics (IMW), Bielefeld University, Universit\"atsstrasse 25, 33615, Bielefeld, Germany.
Our paper concerns the computation of Nash equilibria of first-price auctions with correlated values. While there exist several equilibrium computation methods for auctions with independent values, the correlation of the bidders' values introduces significant complications that render existing methods unsatisfactory in practice. Our contribution is a step towards filling this gap: inspired by the seminal fictitious play process of Brown and Robinson, we present a learning heuristic-that we call fictitious bidding (FB)-for estimating Bayes-Nash equilibria of first-price auctions with correlated…
We construct an approximate public-signal correlated equilibrium for a nonzero-sum differential game in the class of stochastic strategies with memory. The construction is based on a solution of an auxiliary nonzero-sum continuous-time stochastic game. This class of games includes stochastic differential games and continuous-time Markov games. Moreover, we study the limit of approximate equilibrium outcomes in the case when the auxiliary stochastic games tend to the original deterministic one. We show that it lies in the convex hull of the set of equilibrium values provided by deterministic pu…
We investigate the consequences of allowing players to adopt strategies which take advantage of quantum randomization devices. In games of full information, the resulting equilibria are always correlated equilibria, but not all correlated equilibria appear as quantum equilibria. The classical and quantum theories diverge further in games of private information. In the quantum context, we show that Kuhn's equivalence between behavioral and mixed strategies breaks down. As a result, quantum technology allows players to achieve outcomes that would not be achievable with any classical technology s…
This document consists of two parts: the second part was submitted earlier as a new proof of Nash's theorem, and the first part is a note explaining a problem found in that proof. We are indebted to Sergiu Hart and Eran Shmaya for their careful study which led to their simultaneous discovery of this error. So far the error has not been fixed, but many of the results and techniques of the paper remain valid, so we will continue to make it available online. Abstract for the original paper: We give a novel proof of the existence of Nash equilibria in all finite games without using fixed point the…
The dual reduction process, introduced by Myerson, allows to reduce a finite game into a smaller dimensional game such that any equilibrium of the reduced game is an equilibrium of the original game. This holds both for Nash equilibrium and correlated equilibrium. We present examples of applications of dual reduction and argue that this is a useful tool to study Nash equilibria and correlated equilibria. We then investigate its properties.
The existence of stationary Markov perfect equilibria in stochastic games is shown under a general condition called "(decomposable) coarser transition kernels". This result covers various earlier existence results on correlated equilibria, noisy stochastic games, stochastic games with finite actions and state-independent transitions, and stochastic games with mixtures of constant transition kernels as special cases. A remarkably simple proof is provided via establishing a new connection between stochastic games and conditional expectations of correspondences. New applications of stochastic gam…
Many emerging problems involve teams of agents taking part in a game. Such problems require a stochastic analysis with regard to the correlation structures among the agents belonging to a given team. In the context of Standard Borel spaces, this paper makes the following contributions for two teams of finitely many agents taking part in a zero-sum game: (i) An existence result will be presented for saddle-point equilibria in zero-sum games involving teams against teams when common randomness is assumed to be available in each team with an analysis on conditions for compactness of strategic tea…
We investigate whether having a unique equilibrium (or a given number of equilibria) is robust to perturbation of the payoffs, both for Nash equilibrium and correlated equilibrium. We show that the set of n-player finite games with a unique correlated equilibrium is open, while this is not true of Nash equilibrium for n>2. The crucial lemma is that a unique correlated equilibrium is a quasi-strict Nash equilibrium. Related results are studied. For instance, we show that generic two-person zero-sum games have a unique correlated equilibrium and that, while the set of symmetric bimatrix games wi…
Columbia University（哥伦比亚大学）；National Technical University of Athens（雅典国立技术大学）；Univ. Grenoble Alpes（格勒诺布尔阿尔卑斯大学）；Criteo AI Lab（Criteo人工智能实验室）；Singapore University of Technology and Design（新加坡科技设计大学）
Understanding the behavior of no-regret dynamics in general $N$-player games is a fundamental question in online learning and game theory. A folk result in the field states that, in finite games, the empirical frequency of play under no-regret learning converges to the game's set of coarse correlated equilibria. By contrast, our understanding of how the day-to-day behavior of the dynamics correlates to the game's Nash equilibria is much more limited, and only partial results are known for certain classes of games (such as zero-sum or congestion games). In this paper, we study the dynamics of "…
Self-play methods based on regret minimization have become the state of the art for computing Nash equilibria in large two-players zero-sum extensive-form games. These methods fundamentally rely on the hierarchical structure of the players' sequential strategy spaces to construct a regret minimizer that recursively minimizes regret at each decision point in the game tree. In this paper, we introduce the first efficient regret minimization algorithm for computing extensive-form correlated equilibria in large two-player general-sum games with no chance moves. Designing such an algorithm is signi…
Department of IEOR, Columbia University（哥伦比亚大学工业工程与运营研究系）；Mathematical Institute, University of Oxford（牛津大学数学系）；Citadel Securities（Citadel证券公司）
Reinforcement learning for multi-agent games has attracted lots of attention recently. However, given the challenge of solving Nash equilibria for large population games, existing works with guaranteed polynomial complexities either focus on variants of zero-sum and potential games, or aim at solving (coarse) correlated equilibria, or require access to simulators, or rely on certain assumptions that are hard to verify. This work proposes MF-OML (Mean-Field Occupation-Measure Learning), an online mean-field reinforcement learning algorithm for computing approximate Nash equilibria of large popu…
