We exhibit the rich structure of the set of correlated equilibria by analyzing the simplest of polynomial games: the mixed extension of matching pennies. We show that while the correlated equilibrium set is convex and compact, the structure of its extreme points can be quite complicated. In finite games the ratio of extreme correlated to extreme Nash equilibria can be greater than exponential in the size of the strategy spaces. In polynomial games there can exist extreme correlated equilibria which are not finitely supported; we construct a large family of examples using techniques from ergodic theory. We show that in general the set of correlated equilibrium distributions of a polynomial game cannot be described by conditions on finitely many moments (means, covariances, etc.), in marked contrast to the set of Nash equilibria which is always expressible in terms of finitely many moments.
It is known that the set of all correlated equilibria of an n-player non-cooperative game is a convex polytope and includes all the Nash equilibria. Further, the Nash equilibria all lie on the boundary of this polytope. We study the geometry of both these equilibrium notions when the players have cumulative prospect theoretic (CPT) preferences. The set of CPT correlated equilibria includes all the CPT Nash equilibria but it need not be a convex polytope. We show that it can, in fact, be disconnected. However, all the CPT Nash equilibria continue to lie on its boundary. We also characterize the…
The majority of distributed learning literature focuses on convergence to Nash equilibria. Correlated equilibria, on the other hand, can often characterize more efficient collective behavior than even the best Nash equilibrium. However, there are no existing distributed learning algorithms that converge to specific correlated equilibria. In this paper, we provide one such algorithm which guarantees that the agents' collective joint strategy will constitute an efficient correlated equilibrium with high probability. The key to attaining efficient correlated behavior through distributed learning …
We present several new characterizations of correlated equilibria in games with continuous utility functions. These have the advantage of being more computationally and analytically tractable than the standard definition in terms of departure functions. We use these characterizations to construct effective algorithms for approximating a single correlated equilibrium or the entire set of correlated equilibria of a game with polynomial utility functions.
We study correlated equilibria and coarse equilibria of simple first-price single-item auctions in the simplest auction model of full information. Nash equilibria are known to always yield full efficiency and a revenue that is at least the second-highest value. We prove that the same is true for all correlated equilibria, even those in which agents overbid -- i.e., bid above their values. Coarse equilibria, in contrast, may yield lower efficiency and revenue. We show that the revenue can be as low as 26% of the second-highest value in a coarse equilibrium, even if agents are assumed not to ove…
In this paper, we consider the problem of wireless power control in an interference channel where transmitters aim to maximize their own benefit. When the individual payoff or utility function is derived from the transmission efficiency and the spent power, previous works typically study the Nash equilibrium of the resulting power control game. We propose to introduce concepts of correlated and communication equilibria from game theory to find efficient solutions (compared to the Nash equilibrium) for this problem. Communication and correlated equilibria are analyzed for the power control game…
We introduce the notion of exchangeable equilibria of a symmetric bimatrix game, defined as those correlated equilibria in which players' strategy choices are conditionally independently and identically distributed given some hidden variable. We give several game-theoretic interpretations and a version of the "revelation principle". Geometrically, the set of exchangeable equilibria is convex and lies between the symmetric Nash equilibria and the symmetric correlated equilibria. Exchangeable equilibria can achieve higher expected utility than symmetric Nash equilibria.
We prove the existence of approximate correlated equilibrium of support size polylogarithmic in the number of players and the number of actions per player. In particular, using the probabilistic method, we show that there exists a multiset of polylogarithmic size such that the uniform distribution over this multiset forms an approximate correlated equilibrium. Along similar lines, we establish the existence of approximate coarse correlated equilibrium with logarithmic support. We complement these results by considering the computational complexity of determining small-support approximate equil…
We consider a track selection problem for multi-target tracking in a multifunction radar network from a game-theoretic perspective. The problem is formulated as a non-cooperative game. The radars are considered to be players in this game with utilities modeled using a proper tracking accuracy criterion and their strategies are the observed targets whose number is known. Initially, for the problem of coordination, the Nash equilibria are characterized and, in order to find equilibria points, a distributed algorithm based on the best-response dynamics is proposed. Afterwards, the analysis is ext…
A recent emerging trend in the literature on learning in games has been concerned with providing faster learning dynamics for correlated and coarse correlated equilibria in normal-form games. Much less is known about the significantly more challenging setting of extensive-form games, which can capture both sequential and simultaneous moves, as well as imperfect information. In this paper we establish faster no-regret learning dynamics for \textit{extensive-form correlated equilibria (EFCE)} in multiplayer general-sum imperfect-information extensive-form games. When all players follow our accel…
Game-theoretic techniques and equilibria analysis facilitate the design and verification of competitive systems. While algorithmic complexity of equilibria computation has been extensively studied, practical implementation and application of game-theoretic methods is more recent. Tools such as PRISM-games support automated verification and synthesis of zero-sum and (epsilon-optimal subgame-perfect) social welfare Nash equilibria properties for concurrent stochastic games. However, these methods become inefficient as the number of agents grows and may also generate equilibria that yield signifi…
Our paper concerns the computation of Nash equilibria of first-price auctions with correlated values. While there exist several equilibrium computation methods for auctions with independent values, the correlation of the bidders' values introduces significant complications that render existing methods unsatisfactory in practice. Our contribution is a step towards filling this gap: inspired by the seminal fictitious play process of Brown and Robinson, we present a learning heuristic-that we call fictitious bidding (FB)-for estimating Bayes-Nash equilibria of first-price auctions with correlated…
One of the most appealing aspects of the (coarse) correlated equilibrium concept is that natural dynamics quickly arrive at approximations of such equilibria, even in games with many players. In addition, there exist polynomial-time algorithms that compute exact (coarse) correlated equilibria. In light of these results, a natural question is how good are the (coarse) correlated equilibria that can arise from any efficient algorithm or dynamics. In this paper we address this question, and establish strong negative results. In particular, we show that in multiplayer games that have a succinct re…
Politecnico di Milano（米兰理工大学）；University of Southampton（南安普顿大学）
We investigate the computation of equilibria in extensive-form games where ex ante correlation is possible, focusing on correlated equilibria requiring the least amount of communication between the players and the mediator. Motivated by the hardness results on the computation of normal-form correlated equilibria, we introduce the notion of normal-form coarse correlated equilibrium, extending the definition of coarse correlated equilibrium to sequential games. We show that, in two-player games without chance moves, an optimal (e.g., social welfare maximizing) normal-form coarse correlated equil…
We analyze the problem of computing a correlated equilibrium that optimizes some objective (e.g., social welfare). Papadimitriou and Roughgarden [2008] gave a sufficient condition for the tractability of this problem; however, this condition only applies to a subset of existing representations. We propose a different algorithmic approach for the optimal CE problem that applies to all compact representations, and give a sufficient condition that generalizes that of Papadimitriou and Roughgarden. In particular, we reduce the optimal CE problem to the deviation-adjusted social welfare problem, a …
We show that, for any sufficiently small fixed $ε> 0$, when both players in a general-sum two-player (bimatrix) game employ optimistic mirror descent (OMD) with smooth regularization, learning rate $η= O(ε^2)$ and $T = Ω(\text{poly}(1/ε))$ repetitions, either the dynamics reach an $ε$-approximate Nash equilibrium (NE), or the average correlated distribution of play is an $Ω(\text{poly}(ε))$-strong coarse correlated equilibrium (CCE): any possible unilateral deviation does not only leave the player worse, but will decrease its utility by $Ω(\text{poly}(ε))$. As an immediate consequence, when th…
Simple adaptive procedures that converge to correlated equilibria are known to exist for normal form games (Hart and Mas-Colell 2000), but no such analogue exists for extensive-form games. Leveraging inspiration from Zinkevich et al. (2008), we show that any internal regret minimization procedure designed for normal-form games can be efficiently extended to finite extensive-form games of perfect recall. Our procedure converges to the set of forgiving correlated equilibria, a refinement of various other proposed extensions of the correlated equilibrium solution concept to extensive-form games (…
We investigate the degree of discontinuity of several solution concepts from non-cooperative game theory. While the consideration of Nash equilibria forms the core of our work, also pure and correlated equilibria are dealt with. Formally, we restrict the treatment to two player games, but results and proofs extend to the n-player case. As a side result, the degree of discontinuity of solving systems of linear inequalities is settled.
Quantum game theory lays a foundation for understanding the interaction of people using quantum computers with conflicting interests. Recently Zhang proposed a simple yet rich model to study quantum strategic games, and addressed some quantitative questions for general games of growing sizes \cite{Zha10}. However, one fundamental question that the paper did not consider is the characterization of quantum correlated equilibria (QCE). In this paper, we answer this question by giving a sufficient and necessary condition for an arbitrary state $ρ$ being a QCE. In addition, when the condition fails…
We consider the complexity of finding a correlated equilibrium of an $n$-player game in a model that allows the algorithm to make queries on players' payoffs at pure strategy profiles. Randomized regret-based dynamics are known to yield an approximate correlated equilibrium efficiently, namely, in time that is polynomial in the number of players $n$. Here we show that both randomization and approximation are necessary: no efficient deterministic algorithm can reach even an approximate correlated equilibrium, and no efficient randomized algorithm can reach an exact correlated equilibrium. The r…
Unlike normal-form games, where correlated equilibria have been studied for more than 45 years, extensive-form correlation is still generally not well understood. Part of the reason for this gap is that the sequential nature of extensive-form games allows for a richness of behaviors and incentives that are not possible in normal-form settings. This richness translates to a significantly different complexity landscape surrounding extensive-form correlated equilibria. As of today, it is known that finding an optimal extensive-form correlated equilibrium (EFCE), extensive-form coarse correlated e…
We study constrained general-sum stochastic games with unknown Markovian dynamics. A distributed constrained no-regret Q-learning scheme (CNRQ) is presented to guarantee convergence to the set of stationary correlated equilibria of the game. Prior art addresses the unconstrained case only, is structured with nested control loops, and has no convergence result. CNRQ is cast as a single-loop three-timescale asynchronous stochastic approximation algorithm with set-valued update increments. A rigorous convergence analysis with differential inclusion arguments is given which draws on recent extensi…
We consider $ε$-equilibria notions for constant value of $ε$ in $n$-player $m$-actions games where $m$ is a constant. We focus on the following question: What is the largest grid size over the mixed strategies such that $ε$-equilibrium is guaranteed to exist over this grid. For Nash equilibrium, we prove that constant grid size (that depends on $ε$ and $m$, but not on $n$) is sufficient to guarantee existence of weak approximate equilibrium. This result implies a polynomial (in the input) algorithm for weak approximate equilibrium. For approximate Nash equilibrium we introduce a closely relate…
In game theory, a trusted mediator acting on behalf of the players can enable the attainment of correlated equilibria, which may provide better payoffs than those available from the Nash equilibria alone. We explore the approach of replacing the trusted mediator with an unconditionally secure sampling protocol that jointly generates the players' actions. We characterize the joint distributions that can be securely sampled by malicious players via protocols using error-free communication. This class of distributions depends on whether players may speak simultaneously ("cheap talk") or must spea…
We study the application of iterative first-order methods to the problem of computing equilibria of large-scale two-player extensive-form games. First-order methods must typically be instantiated with a regularizer that serves as a distance-generating function for the decision sets of the players. For the case of two-player zero-sum games, the state-of-the-art theoretical convergence rate for Nash equilibrium is achieved by using the dilated entropy function. In this paper, we introduce a new entropy-based distance-generating function for two-player zero-sum games, and show that this function …
We consider repeated games where the players behave according to cumulative prospect theory (CPT). We show that, when the players have calibrated strategies and behave according to CPT, the natural analog of the notion of correlated equilibrium in the CPT case, as defined by Keskin, is not enough to capture all subsequential limits of the empirical distribution of action play. We define the notion of a mediated CPT correlated equilibrium via an extension of the stage game to a so-called mediated game. We then show, along the lines of the result of Foster and Vohra about convergence to the set …
The Complexity of Computing Coarse Correlated Equilibria in Markov Games with a Single Controller
Massachusetts Institute of Technology（麻省理工学院）；National Technical University of Athens（雅典国立技术大学）；Archimedes, Athena Research Center（阿基米德，雅典研究中心）；University of California, Irvine（加州大学尔湾分校）；National and Kapodistrian University of Athens（雅典国家与卡波迪斯特里阿斯大学）
The designs of many large-scale systems today, from traffic routing environments to smart grids, rely on game-theoretic equilibrium concepts. However, as the size of an $N$-player game typically grows exponentially with $N$, standard game theoretic analysis becomes effectively infeasible beyond a low number of players. Recent approaches have gone around this limitation by instead considering Mean-Field games, an approximation of anonymous $N$-player games, where the number of players is infinite and the population's state distribution, instead of every individual player's state, is the object …
In nature and society problems arise when different interests are difficult to reconcile, which are modeled in game theory. While most applications assume uncorrelated games, a more detailed modeling is necessary to consider the correlations that influence the decisions of the players. The current theory for correlated games, however, enforces the players to obey the instructions from a third party or "correlation device" to reach equilibrium, but this cannot be achieved for all initial correlations. We extend here the existing framework of correlated games and find that there are other intere…
We address an open problem on the computability of correlated equilibria in a variant of polymatrix where each player's utility is the maximum of their edge payoffs. We demonstrate that this max-variant game has the polynomial expectation property, and the results of Papadimitriou and Roughgarden can thus be applied. We propose ideas for extending these findings to other variants of polymatrix games, as well as briefly address the broader question of necessity for the polynomial expectation property when computing correlated equilibria.
In a landmark paper, Papadimitriou and Roughgarden described a polynomial-time algorithm ("Ellipsoid Against Hope") for computing sample correlated equilibria of concisely-represented games. Recently, Stein, Parrilo and Ozdaglar showed that this algorithm can fail to find an exact correlated equilibrium, but can be easily modified to efficiently compute approximate correlated equilibria. Currently, it remains unresolved whether the algorithm can be modified to compute an exact correlated equilibrium. We show that it can, presenting a variant of the Ellipsoid Against Hope algorithm that guarant…
This document consists of two parts: the second part was submitted earlier as a new proof of Nash's theorem, and the first part is a note explaining a problem found in that proof. We are indebted to Sergiu Hart and Eran Shmaya for their careful study which led to their simultaneous discovery of this error. So far the error has not been fixed, but many of the results and techniques of the paper remain valid, so we will continue to make it available online. Abstract for the original paper: We give a novel proof of the existence of Nash equilibria in all finite games without using fixed point the…
