Motivated by uncertain parameters encountered in Markov decision processes (MDPs) and stochastic games, we study the effect of parameter uncertainty on Bellman operator-based algorithms under a set-based framework. Specifically, we first consider a family of MDPs where the cost parameters are in a given compact set; we then define a Bellman operator acting on a set of value functions to produce a new set of value functions as the output under all possible variations in the cost parameter. We prove the existence of a fixed point of this set-based Bellman operator by showing that it is contractive on a complete metric space, and explore its relationship with the corresponding family of MDPs and stochastic games. Additionally, we show that given interval set bounded cost parameters, we can form exact bounds on the set of optimal value functions. Finally, we utilize our results to bound the value function trajectory of a player in a stochastic game.
A bimatrix game $(A,B)$ is called a game of rank $k$ if the rank of the matrix $A+B$ is at most $k$. We consider the problem of enumerating the Nash equilibria in (non-degenerate) games of rank 1. In particular, we show that even for games of rank 1 not all equilibria can be reached by a Lemke-Howson path and present a parametric simplex-type algorithm for enumerating all Nash equilibria of a non-degenerate game of rank 1.
Designing efficient algorithms to compute Nash equilibria poses considerable challenges in Algorithmic Game Theory and Optimization. In this work, we employ integer programming techniques to compute Nash equilibria in Integer Programming Games, a class of simultaneous and non-cooperative games where each player solves a parametrized integer program. We introduce ZERO Regrets, a general and efficient cutting plane algorithm to compute, enumerate, and select Nash equilibria. Our framework leverages the concept of equilibrium inequality, an inequality valid for any Nash equilibrium, and the assoc…
If a game has a unique Nash equilibrium, then this equilibrium is arguably the solution of the game from the refinement's literature point of view. However, it might be that for almost all initial conditions, all strategies in the support of this equilibrium are eliminated by the replicator dynamics and the best-reply dynamics.
Our paper concerns the computation of Nash equilibria of first-price auctions with correlated values. While there exist several equilibrium computation methods for auctions with independent values, the correlation of the bidders' values introduces significant complications that render existing methods unsatisfactory in practice. Our contribution is a step towards filling this gap: inspired by the seminal fictitious play process of Brown and Robinson, we present a learning heuristic-that we call fictitious bidding (FB)-for estimating Bayes-Nash equilibria of first-price auctions with correlated…
This document consists of two parts: the second part was submitted earlier as a new proof of Nash's theorem, and the first part is a note explaining a problem found in that proof. We are indebted to Sergiu Hart and Eran Shmaya for their careful study which led to their simultaneous discovery of this error. So far the error has not been fixed, but many of the results and techniques of the paper remain valid, so we will continue to make it available online. Abstract for the original paper: We give a novel proof of the existence of Nash equilibria in all finite games without using fixed point the…
We prove that differential Nash equilibria are generic amongst local Nash equilibria in continuous zero-sum games. That is, there exists an open-dense subset of zero-sum games for which local Nash equilibria are non-degenerate differential Nash equilibria. The result extends previous results to the zero-sum setting, where we obtain even stronger results; in particular, we show that local Nash equilibria are generically hyperbolic critical points. We further show that differential Nash equilibria of zero-sum games are structurally stable. The purpose for presenting these extensions is the recen…
It is known that the set of all correlated equilibria of an n-player non-cooperative game is a convex polytope and includes all the Nash equilibria. Further, the Nash equilibria all lie on the boundary of this polytope. We study the geometry of both these equilibrium notions when the players have cumulative prospect theoretic (CPT) preferences. The set of CPT correlated equilibria includes all the CPT Nash equilibria but it need not be a convex polytope. We show that it can, in fact, be disconnected. However, all the CPT Nash equilibria continue to lie on its boundary. We also characterize the…
In this paper, we compute $ε$-approximate Nash equilibria in atomic splittable polymatroid congestion games with convex Lipschitz continuous cost functions. The main approach relies on computing a pure Nash equilibrium for an associated integrally-splittable congestion game, where players can only split their demand in integral multiples of a common packet size. It is known that one can compute pure Nash equilibria for integrally-splittable congestion games within a running time that is pseudo-polynomial in the aggregated demand of the players. As the main contribution of this paper, we decide…
An heuristic approach to compute strong Nash (Aumann) equilibria is presented. The method is based on differential evolution and three variants of a generative relation for strong Nash equilibria characterization. Numerical experiments performed on the minimum effort game for up to 150 players illustrate the efficiency of the approach. The advantages and disadvantages of each variant is discussed in terms of precision and running time.
We study pure Nash equilibria in infinite-duration games on graphs, with partial visibility of actions but communication (based on a graph) among the players. We show that a simple communication mechanism consisting in reporting the deviator when seeing it and propagating this information is sufficient for characterizing Nash equilibria. We propose an epistemic game construction, which conveniently records important information about the knowledge of the players. With this abstraction, we are able to characterize Nash equilibria which follow the simple communication pattern via winning strateg…
We develop a quasi-polynomial time Las Vegas algorithm for approximating Nash equilibria in polymatrix games over trees, under a mild renormalizing assumption. Our result, in particular, leads to an expected polynomial-time algorithm for computing approximate Nash equilibria of tree polymatrix games in which the number of actions per player is a fixed constant. Further, for trees with constant degree, the running time of the algorithm matches the best known upper bound for approximating Nash equilibria in bimatrix games (Lipton, Markakis, and Mehta 2003). Notably, this work closely complements…
The rank of a bimatrix game (A,B) is the rank of the matrix A+B. We give a construction of rank-1 games with exponentially many equilibria, which answers an open problem by Kannan and Theobald (2010).
We show that the problem of finding an ε-approximate Nash equilibrium in an anonymous game with seven pure strategies is complete in PPAD, when the approximation parameter ε is exponentially small in the number of players.
We study a static game played by a finite number of agents, in which agents are assigned independent and identically distributed random types and each agent minimizes its objective function by choosing from a set of admissible actions that depends on its type. The game is anonymous in the sense that the objective function of each agent depends on the actions of other agents only through the empirical distribution of their type-action pairs. We study the asymptotic behavior of Nash equilibria, as the number of agents tends to infinity, first by deriving laws of large numbers characterizes almos…
We investigate the degree of discontinuity of several solution concepts from non-cooperative game theory. While the consideration of Nash equilibria forms the core of our work, also pure and correlated equilibria are dealt with. Formally, we restrict the treatment to two player games, but results and proofs extend to the n-player case. As a side result, the degree of discontinuity of solving systems of linear inequalities is settled.
We study the query complexity of approximate notions of Nash equilibrium in games with a large number of players $n$. Our main result states that for $n$-player binary-action games and for constant $\varepsilon$, the query complexity of an $\varepsilon$-well-supported Nash equilibrium is exponential in $n$. One of the consequences of this result is an exponential lower bound on the rate of convergence of adaptive dynamics to approxiamte Nash equilibrium.
In two player bi-matrix games with partial monitoring, actions played are not observed, only some messages are received. Those games satisfy a crucial property of usual bi-matrix games: there are only a finite number of required (mixed) best replies. This is very helpful while investigating sets of Nash equilibria: for instance, in some cases, it allows to relate it to the set of equilibria of some auxiliary game with full monitoring. In the general case, the Lemke-Howson algorithm is extended and, under some genericity assumption, its output are Nash equilibria of the original game. As a by p…
We introduce Q-Nash, a quantum annealing algorithm for the NP-complete problem of Fnding pure Nash equilibria in graphical games. The algorithm consists of two phases. The first phase determines all combinations of best response strategies for each player using classical computation. The second phase finds pure Nash equilibria using a quantum annealing device by mapping the computed combinations to a quadratic unconstrained binary optimization formulation based on the Set Cover problem. We empirically evaluate Q-Nash on D-Wave's Quantum Annealer 2000Q using different graphical game topologies.…
We investigate the complexity of bounding the uncertainty of graphical games, and we provide new insight into the intrinsic difficulty of computing Nash equilibria. In particular, we show that, if one adds very simple and natural additional requirements to a graphical game, the existence of Nash equilibria is no longer guaranteed, and computing an equilibrium is an intractable problem. Moreover, if stronger equilibrium conditions are required for the game, we get hardness results for the second level of the polynomial hierarchy. Our results offer a clear picture of the complexity of mixed Nash…
We compute a parametric description of the totally mixed Nash equilibria of a generic game in normal form with pre-fixed structure. Using this representation, we show conditions under which a game has the maximum possible number of this kind of equilibria. Then, we present a symbolic procedure that allows us to describe and estimate the number of isolated totally mixed Nash equilibria of an arbitrary game. Under certain assumptions, the algorithm computes the exact number of these equilibria.
Observable games are game situations that reach one of possibly many Nash equilibria. Before an instance of the game starts, an external observer does not know, a priori, what is the exact profile of actions that will occur; thus, he assigns subjective probabilities to players' actions. However, not all probabilistic assignments are coherent with a given game. We study the decision problem of determining if a given set of probabilistic constraints assigned a priori by the observer to a given game is coherent, which we call the Coherence of Probabilistic Constraints on Equilibria, or PCE-Cohere…
We prove that in a normal form n-player game with m actions for each player, there exists an approximate Nash equilibrium where each player randomizes uniformly among a set of O(log(m) + log(n)) pure strategies. This result induces an $N^{\log \log N}$ algorithm for computing an approximate Nash equilibrium in games where the number of actions is polynomial in the number of players (m=poly(n)), where $N=nm^n$ is the size of the game (the input size). In addition, we establish an inverse connection between the entropy of Nash equilibria in the game, and the time it takes to find such an approxi…
Computing Nash equilibria for strategic multi-agent systems is challenging for expensive black box systems. Motivated by the ubiquity of games involving exploitation of common resources, this paper considers the above problem for potential games. We use the Bayesian optimization framework to obtain novel algorithms to solve finite (discrete action spaces) and infinite (real interval action spaces) potential games, utilizing the structure of potential games. Numerical results illustrate the efficiency of the approach in computing the Nash equilibria of static potential games and linear Nash equ…
We prove an $N^{2-o(1)}$ lower bound on the randomized communication complexity of finding an $ε$-approximate Nash equilibrium (for constant $ε>0$) in a two-player $N\times N$ game.
University of Aberdeen（阿伯丁大学）；University of Artois（阿尔图瓦大学）
In this paper we describe an argumentation-based representation of normal form games, and demonstrate how argumentation can be used to compute pure strategy Nash equilibria. Our approach builds on Modgil's Extended Argumentation Frameworks. We demonstrate its correctness, prove several theoretical properties it satisfies, and outline how it can be used to explain why certain strategies are Nash equilibria to a non-expert human user.
In this paper we present a novel generic mapping between Graphical Games and Markov Random Fields so that pure Nash equilibria in the former can be found by statistical inference on the latter. Thus, the problem of deciding whether a graphical game has a pure Nash equilibrium, a well-known intractable problem, can be attacked by well-established algorithms such as Belief Propagation, Junction Trees, Markov Chain Monte Carlo and Simulated Annealing. Large classes of graphical games become thus tractable, including all classes already known, but also new classes such as the games with O(log n) t…
Given a multifunction from $X$ to the $k-$fold symmetric product $Sym_k(X)$, we use the Dold-Thom Theorem to establish a homological selection Theorem. This is used to establish existence of Nash equilibria. Cost functions in problems concerning the existence of Nash Equilibria are traditionally multilinear in the mixed strategies. The main aim of this paper is to relax the hypothesis of multilinearity. We use basic intersection theory, Poincaré Duality in addition to the Dold-Thom Theorem.
We study the complexity of computing a uniform Nash equilibrium on a non-win-lose bimatrix game. It is known that such a problem is NP-complete even if a bimatrix game is win-lose (Bonifaci et al., 2008). Fortunately, if a win-lose bimatrix game is planar, then uniform Nash equilibria always exist. We have a polynomial-time algorithm for finding a uniform Nash equilibrium of a planar win-lose bimatrix game (Addario-Berry et al., 2007). The following question is left: How hard to compute a uniform Nash equilibrium on a planar non-win-lose bimatrix game? This paper resolves this issue. We prove …
We study the problem of checking for the existence of constrained pure Nash equilibria in a subclass of polymatrix games defined on weighted directed graphs. The payoff of a player is defined as the sum of nonnegative rational weights on incoming edges from players who picked the same strategy augmented by a fixed integer bonus for picking a given strategy. These games capture the idea of coordination within a local neighbourhood in the absence of globally common strategies. We study the decision problem of checking whether a given set of strategy choices for a subset of the players is consist…
In an epsilon-Nash equilibrium, a player can gain at most epsilon by changing his behaviour. Recent work has addressed the question of how best to compute epsilon-Nash equilibria, and for what values of epsilon a polynomial-time algorithm exists. An epsilon-well-supported Nash equilibrium (epsilon-WSNE) has the additional requirement that any strategy that is used with non-zero probability by a player must have payoff at most epsilon less than the best response. A recent algorithm of Kontogiannis and Spirakis shows how to compute a 2/3-WSNE in polynomial time, for bimatrix games. Here we intro…
MID-LIDS；MIT-IBM Watson AI Lab（麻省理工学院-IBM沃森人工智能实验室）；IBM-Research（IBM研究院）；Mila（米拉）；University of Oxford（牛津大学）
Multiagent learning settings are inherently more difficult than single-agent learning because each agent interacts with other simultaneously learning agents in a shared environment. An effective approach in multiagent reinforcement learning is to consider the learning process of agents and influence their future policies toward desirable behaviors from each agent's perspective. Importantly, if each agent maximizes its long-term rewards by accounting for the impact of its behavior on the set of convergence policies, the resulting multiagent system reaches an active equilibrium. While this new s…
The $\varepsilon$-well-supported Nash equilibrium is a strong notion of approximation of a Nash equilibrium, where no player has an incentive greater than $\varepsilon$ to deviate from any of the pure strategies that she uses in her mixed strategy. The smallest constant $\varepsilon$ currently known for which there is a polynomial-time algorithm that computes an $\varepsilon$-well-supported Nash equilibrium in bimatrix games is slightly below $2/3$. In this paper we study this problem for symmetric bimatrix games and we provide a polynomial-time algorithm that gives a $(1/2+δ)$-well-supported …
We study the problem of finding approximate Nash equilibria that satisfy certain conditions, such as providing good social welfare. In particular, we study the problem $ε$-NE $δ$-SW: find an $ε$-approximate Nash equilibrium ($ε$-NE) that is within $δ$ of the best social welfare achievable by an $ε$-NE. Our main result is that, if the exponential-time hypothesis (ETH) is true, then solving $\left(\frac{1}{8} - \mathrm{O}(δ)\right)$-NE $\mathrm{O}(δ)$-SW for an $n\times n$ bimatrix game requires $n^{\mathrm{\widetilde Ω}(\log n)}$ time. Building on this result, we show similar conditional runnin…
We study the problem of computing approximate Nash equilibria of bimatrix games, in a setting where players initially know their own payoffs but not the payoffs of the other player. In order for a solution of reasonable quality to be found, some amount of communication needs to take place between the players. We are interested in algorithms where the communication is substantially less than the contents of a payoff matrix, for example logarithmic in the size of the matrix. When the communication is polylogarithmic in the number of strategies n, we show how to obtain epsilon-approximate Nash eq…
We study the deterministic and randomized query complexity of finding approximate equilibria in bimatrix games. We show that the deterministic query complexity of finding an $ε$-Nash equilibrium when $ε< \frac{1}{2}$ is $Ω(k^2)$, even in zero-one constant-sum games. In combination with previous results \cite{FGGS13}, this provides a complete characterization of the deterministic query complexity of approximate Nash equilibria. We also study randomized querying algorithms. We give a randomized algorithm for finding a $(\frac{3 - \sqrt{5}}{2} + ε)$-Nash equilibrium using $O(\frac{k \cdot \log k}…
We investigate a model for representing large multiplayer games, which satisfy strong symmetry properties. This model is made of multiple copies of an arena; each player plays in his own arena, and can partially observe what the other players do. Therefore, this game has partial information and symmetry constraints, which make the computation of Nash equilibria difficult. We show several undecidability results, and for bounded-memory strategies, we precisely characterize the complexity of computing pure Nash equilibria (for qualitative objectives) in this game model.
The problem of finding pure strategy Nash equilibria in multiagent concurrent games with finite-horizon temporal goals has received some recent attention. Earlier work solved this problem through the use of Rabin automata. In this work, we take advantage of the finite-horizon nature of the agents' goals and show that checking for and finding pure strategy Nash equilibria can be done using a combination of safety games and lasso testing in Büchi automata. To separate strategic reasoning from temporal reasoning, we model agents' goals by deterministic finite-word automata (DFAs), since finite-ho…
We study the computational complexity of Nash equilibria in concurrent games with limit-average objectives. In particular, we prove that the existence of a Nash equilibrium in randomised strategies is undecidable, while the existence of a Nash equilibrium in pure strategies is decidable, even if we put a constraint on the payoff of the equilibrium. Our undecidability result holds even for a restricted class of concurrent games, where nonzero rewards occur only on terminal states. Moreover, we show that the constrained existence problem is undecidable not only for concurrent games but for turn-…
We discuss and solve a model for a game with many players, where a subset of truely deciding players is embedded into a hierarchy of dependent agents. These interdependencies modify the game matrix and the Nash equilibria for the deciding players. In a concrete example, we recognize the partition function of the Ising model and for high dependency we observe a phase transition to a new Nash equilibrium, which is the Pareto-efficient outcome. An example we have in mind is the game theory for major shareholders in a stock market, where intermediate companies decide according to a majority vote o…
We provide an in-depth study of Nash equilibria in multi-objective normal form games (MONFGs), i.e., normal form games with vectorial payoffs. Taking a utility-based approach, we assume that each player's utility can be modelled with a utility function that maps a vector to a scalar utility. In the case of a mixed strategy, it is meaningful to apply such a scalarisation both before calculating the expectation of the payoff vector as well as after. This distinction leads to two optimisation criteria. With the first criterion, players aim to optimise the expected value of their utility function …
Since the celebrated PPAD-completeness result for Nash equilibria in bimatrix games, a long line of research has focused on polynomial-time algorithms that compute $\varepsilon$-approximate Nash equilibria. Finding the best possible approximation guarantee that we can have in polynomial time has been a fundamental and non-trivial pursuit on settling the complexity of approximate equilibria. Despite a significant amount of effort, the algorithm of Tsaknakis and Spirakis, with an approximation guarantee of $(0.3393+δ)$, remains the state of the art over the last 15 years. In this paper, we propo…
This paper is an exposition of algorithms for finding one or all equilibria of a bimatrix game (a two-player game in strategic form) in the style of a chapter in a graduate textbook. Using labeled "best-response polytopes", we present the Lemke-Howson algorithm that finds one equilibrium. We show that the path followed by this algorithm has a direction, and that the endpoints of the path have opposite index, in a canonical way using determinants. For reference, we prove that a number of notions of nondegeneracy of a bimatrix game are equivalent. The computation of all equilibria of a general b…
The Colonel Blotto Problem proposed by Borel in 1921 has served as a widely applicable model of budget-constrained simultaneous winner-take-all competitions in the social sciences. Applications include elections, advertising, R&D and more. However, the classic Blotto problem and variants limit the study to competitions over a finite set of discrete battlefields. In this paper, we extend the classical theory to study multiplayer Blotto games over arbitrary measurable battlegrounds, provide an algorithm to efficiently sample equilibria of symmetric "equipartionable" Generalized Blotto games, and…
This position paper presents a synthesis viewpoint of blockchains from two orthogonal perspectives: fault-tolerant distributed systems and game theory. Specifically, we formulate a new game-theoretical problem in the context of blockchains and sketch a closed-form Nash equilibrium to the problem.
This paper concerns computing approximate pure Nash equilibria in weighted congestion games, which has been shown to be PLS-complete. With the help of $\hatΨ$-game and approximate potential functions, we propose two algorithms based on best response dynamics, and prove that they efficiently compute $\fracρ{1-ε}$-approximate pure Nash equilibria for $ρ= d!$ and $ρ=\frac{2\cdot W\cdot(d+1)}{2\cdot W+d+1}\le {d + 1}$, respectively, when the weighted congestion game has polynomial latency functions of degree at most $d \ge 1$ and players' weights are bounded from above by a constant $W \ge 1$. Thi…
