Although it has been known since the 1970s that a globally optimal strategy profile in a common-payoff game is a Nash equilibrium, global optimality is a strict requirement that limits the result's applicability. In this work, we show that any locally optimal symmetric strategy profile is also a (global) Nash equilibrium. Furthermore, we show that this result is robust to perturbations to the common payoff and to the local optimum. Applied to machine learning, our result provides a global guarantee for any gradient method that finds a local optimum in symmetric strategy space. While this result indicates stability to unilateral deviation, we nevertheless identify broad classes of games where mixed local optima are unstable under joint, asymmetric deviations. We analyze the prevalence of instability by running learning algorithms in a suite of symmetric games, and we conclude by discussing the applicability of our results to multi-agent RL, cooperative inverse RL, and decentralized POMDPs.
University of Aberdeen（阿伯丁大学）；University of Artois（阿尔图瓦大学）
In this paper we describe an argumentation-based representation of normal form games, and demonstrate how argumentation can be used to compute pure strategy Nash equilibria. Our approach builds on Modgil's Extended Argumentation Frameworks. We demonstrate its correctness, prove several theoretical properties it satisfies, and outline how it can be used to explain why certain strategies are Nash equilibria to a non-expert human user.
MID-LIDS；MIT-IBM Watson AI Lab（麻省理工学院-IBM沃森人工智能实验室）；IBM-Research（IBM研究院）；Mila（米拉）；University of Oxford（牛津大学）
Multiagent learning settings are inherently more difficult than single-agent learning because each agent interacts with other simultaneously learning agents in a shared environment. An effective approach in multiagent reinforcement learning is to consider the learning process of agents and influence their future policies toward desirable behaviors from each agent's perspective. Importantly, if each agent maximizes its long-term rewards by accounting for the impact of its behavior on the set of convergence policies, the resulting multiagent system reaches an active equilibrium. While this new s…
Reinforcement learning has been shown to be an effective strategy for automatically training policies for challenging control problems. Focusing on non-cooperative multi-agent systems, we propose a novel reinforcement learning framework for training joint policies that form a Nash equilibrium. In our approach, rather than providing low-level reward functions, the user provides high-level specifications that encode the objective of each agent. Then, guided by the structure of the specifications, our algorithm searches over policies to identify one that provably forms an $ε$-Nash equilibrium (wi…
This position paper presents a synthesis viewpoint of blockchains from two orthogonal perspectives: fault-tolerant distributed systems and game theory. Specifically, we formulate a new game-theoretical problem in the context of blockchains and sketch a closed-form Nash equilibrium to the problem.
ETH Zurich（苏黎世联邦理工学院）；University of Toronto（多伦多大学）
Model-free learning for multi-agent stochastic games is an active area of research. Existing reinforcement learning algorithms, however, are often restricted to zero-sum games, and are applicable only in small state-action spaces or other simplified settings. Here, we develop a new data efficient Deep-Q-learning methodology for model-free learning of Nash equilibria for general-sum stochastic games. The algorithm uses a local linear-quadratic expansion of the stochastic game, which leads to analytically solvable optimal actions. The expansion is parametrized by deep neural networks to give it …
Mastercard Corporation（万事达卡公司）；Georgia Institute of Technology（佐治亚理工学院）
We explore the use of policy approximations to reduce the computational cost of learning Nash equilibria in zero-sum stochastic games. We propose a new Q-learning type algorithm that uses a sequence of entropy-regularized soft policies to approximate the Nash policy during the Q-function updates. We prove that under certain conditions, by updating the regularized Q-function, the algorithm converges to a Nash equilibrium. We also demonstrate the proposed algorithm's ability to transfer previous training experiences, enabling the agents to adapt quickly to new environments. We provide a dynamic …
Computing Nash equilibria for strategic multi-agent systems is challenging for expensive black box systems. Motivated by the ubiquity of games involving exploitation of common resources, this paper considers the above problem for potential games. We use the Bayesian optimization framework to obtain novel algorithms to solve finite (discrete action spaces) and infinite (real interval action spaces) potential games, utilizing the structure of potential games. Numerical results illustrate the efficiency of the approach in computing the Nash equilibria of static potential games and linear Nash equ…
We prove that differential Nash equilibria are generic amongst local Nash equilibria in continuous zero-sum games. That is, there exists an open-dense subset of zero-sum games for which local Nash equilibria are non-degenerate differential Nash equilibria. The result extends previous results to the zero-sum setting, where we obtain even stronger results; in particular, we show that local Nash equilibria are generically hyperbolic critical points. We further show that differential Nash equilibria of zero-sum games are structurally stable. The purpose for presenting these extensions is the recen…
It is known that the set of all correlated equilibria of an n-player non-cooperative game is a convex polytope and includes all the Nash equilibria. Further, the Nash equilibria all lie on the boundary of this polytope. We study the geometry of both these equilibrium notions when the players have cumulative prospect theoretic (CPT) preferences. The set of CPT correlated equilibria includes all the CPT Nash equilibria but it need not be a convex polytope. We show that it can, in fact, be disconnected. However, all the CPT Nash equilibria continue to lie on its boundary. We also characterize the…
We provide an in-depth study of Nash equilibria in multi-objective normal form games (MONFGs), i.e., normal form games with vectorial payoffs. Taking a utility-based approach, we assume that each player's utility can be modelled with a utility function that maps a vector to a scalar utility. In the case of a mixed strategy, it is meaningful to apply such a scalarisation both before calculating the expectation of the payoff vector as well as after. This distinction leads to two optimisation criteria. With the first criterion, players aim to optimise the expected value of their utility function …
We analyze the robustness of (pure strategy) Nash equilibria for network games against perturbations of the players' utility functions. We first derive a simple characterization of the margin of robustness, defined as the minimum magnitude of a perturbation that makes a Nash equilibrium of the original game stop being so in the perturbed game. Then, we investigate what the maximally robust equilibria are in some standard network games such as the coordination and the anti-coordination game. Finally, as an application, we provide some sufficient conditions for the existence of Nash equilibria i…
An heuristic approach to compute strong Nash (Aumann) equilibria is presented. The method is based on differential evolution and three variants of a generative relation for strong Nash equilibria characterization. Numerical experiments performed on the minimum effort game for up to 150 players illustrate the efficiency of the approach. The advantages and disadvantages of each variant is discussed in terms of precision and running time.
We study pure Nash equilibria in infinite-duration games on graphs, with partial visibility of actions but communication (based on a graph) among the players. We show that a simple communication mechanism consisting in reporting the deviator when seeing it and propagating this information is sufficient for characterizing Nash equilibria. We propose an epistemic game construction, which conveniently records important information about the knowledge of the players. With this abstraction, we are able to characterize Nash equilibria which follow the simple communication pattern via winning strateg…
We develop a quasi-polynomial time Las Vegas algorithm for approximating Nash equilibria in polymatrix games over trees, under a mild renormalizing assumption. Our result, in particular, leads to an expected polynomial-time algorithm for computing approximate Nash equilibria of tree polymatrix games in which the number of actions per player is a fixed constant. Further, for trees with constant degree, the running time of the algorithm matches the best known upper bound for approximating Nash equilibria in bimatrix games (Lipton, Markakis, and Mehta 2003). Notably, this work closely complements…
The rank of a bimatrix game (A,B) is the rank of the matrix A+B. We give a construction of rank-1 games with exponentially many equilibria, which answers an open problem by Kannan and Theobald (2010).
We show that the problem of finding an ε-approximate Nash equilibrium in an anonymous game with seven pure strategies is complete in PPAD, when the approximation parameter ε is exponentially small in the number of players.
We prove an $N^{2-o(1)}$ lower bound on the randomized communication complexity of finding an $ε$-approximate Nash equilibrium (for constant $ε>0$) in a two-player $N\times N$ game.
We investigate the degree of discontinuity of several solution concepts from non-cooperative game theory. While the consideration of Nash equilibria forms the core of our work, also pure and correlated equilibria are dealt with. Formally, we restrict the treatment to two player games, but results and proofs extend to the n-player case. As a side result, the degree of discontinuity of solving systems of linear inequalities is settled.
Designing efficient algorithms to compute Nash equilibria poses considerable challenges in Algorithmic Game Theory and Optimization. In this work, we employ integer programming techniques to compute Nash equilibria in Integer Programming Games, a class of simultaneous and non-cooperative games where each player solves a parametrized integer program. We introduce ZERO Regrets, a general and efficient cutting plane algorithm to compute, enumerate, and select Nash equilibria. Our framework leverages the concept of equilibrium inequality, an inequality valid for any Nash equilibrium, and the assoc…
We consider multi-agent decision making where each agent's cost function depends on all agents' strategies. We propose a distributed algorithm to learn a Nash equilibrium, whereby each agent uses only obtained values of her cost function at each joint played action, lacking any information of the functional form of her cost or other agents' costs or strategy sets. In contrast to past work where convergent algorithms required strong monotonicity, we prove algorithm convergence under mere monotonicity assumption. This significantly widens algorithm's applicability, such as to games with linear c…
We study the query complexity of approximate notions of Nash equilibrium in games with a large number of players $n$. Our main result states that for $n$-player binary-action games and for constant $\varepsilon$, the query complexity of an $\varepsilon$-well-supported Nash equilibrium is exponential in $n$. One of the consequences of this result is an exponential lower bound on the rate of convergence of adaptive dynamics to approxiamte Nash equilibrium.
We reconsider the training objective of Generative Adversarial Networks (GANs) from the mixed Nash Equilibria (NE) perspective. Inspired by the classical prox methods, we develop a novel algorithmic framework for GANs via an infinite-dimensional two-player game and prove rigorous convergence rates to the mixed NE, resolving the longstanding problem that no provably convergent algorithm exists for general GANs. We then propose a principled procedure to reduce our novel prox methods to simple sampling routines, leading to practically efficient algorithms. Finally, we provide experimental evidenc…
In two player bi-matrix games with partial monitoring, actions played are not observed, only some messages are received. Those games satisfy a crucial property of usual bi-matrix games: there are only a finite number of required (mixed) best replies. This is very helpful while investigating sets of Nash equilibria: for instance, in some cases, it allows to relate it to the set of equilibria of some auxiliary game with full monitoring. In the general case, the Lemke-Howson algorithm is extended and, under some genericity assumption, its output are Nash equilibria of the original game. As a by p…
We introduce Q-Nash, a quantum annealing algorithm for the NP-complete problem of Fnding pure Nash equilibria in graphical games. The algorithm consists of two phases. The first phase determines all combinations of best response strategies for each player using classical computation. The second phase finds pure Nash equilibria using a quantum annealing device by mapping the computed combinations to a quadratic unconstrained binary optimization formulation based on the Set Cover problem. We empirically evaluate Q-Nash on D-Wave's Quantum Annealer 2000Q using different graphical game topologies.…
Facebook AI Research（脸书人工智能研究院）；Ecole Polytechnique Fédérale de Lausanne（洛桑联邦理工学院）；Université Paris-Dauphine（巴黎第九大学）；ENSAE（法国国立统计与经济管理学校）
This paper tackles the problem of adversarial examples from a game theoretic point of view. We study the open question of the existence of mixed Nash equilibria in the zero-sum game formed by the attacker and the classifier. While previous works usually allow only one player to use randomized strategies, we show the necessity of considering randomization for both the classifier and the attacker. We demonstrate that this game has no duality gap, meaning that it always admits approximate Nash equilibria. We also provide the first optimization algorithms to learn a mixture of classifiers that app…
We investigate the complexity of bounding the uncertainty of graphical games, and we provide new insight into the intrinsic difficulty of computing Nash equilibria. In particular, we show that, if one adds very simple and natural additional requirements to a graphical game, the existence of Nash equilibria is no longer guaranteed, and computing an equilibrium is an intractable problem. Moreover, if stronger equilibrium conditions are required for the game, we get hardness results for the second level of the polynomial hierarchy. Our results offer a clear picture of the complexity of mixed Nash…
Observable games are game situations that reach one of possibly many Nash equilibria. Before an instance of the game starts, an external observer does not know, a priori, what is the exact profile of actions that will occur; thus, he assigns subjective probabilities to players' actions. However, not all probabilistic assignments are coherent with a given game. We study the decision problem of determining if a given set of probabilistic constraints assigned a priori by the observer to a given game is coherent, which we call the Coherence of Probabilistic Constraints on Equilibria, or PCE-Cohere…
We prove that in a normal form n-player game with m actions for each player, there exists an approximate Nash equilibrium where each player randomizes uniformly among a set of O(log(m) + log(n)) pure strategies. This result induces an $N^{\log \log N}$ algorithm for computing an approximate Nash equilibrium in games where the number of actions is polynomial in the number of players (m=poly(n)), where $N=nm^n$ is the size of the game (the input size). In addition, we establish an inverse connection between the entropy of Nash equilibria in the game, and the time it takes to find such an approxi…
If a game has a unique Nash equilibrium, then this equilibrium is arguably the solution of the game from the refinement's literature point of view. However, it might be that for almost all initial conditions, all strategies in the support of this equilibrium are eliminated by the replicator dynamics and the best-reply dynamics.
Carnegie Mellon University（卡内基梅隆大学）；University of Waterloo（滑铁卢大学）
Adversarial training is a standard technique for training adversarially robust models. In this paper, we study adversarial training as an alternating best-response strategy in a 2-player zero-sum game. We prove that even in a simple scenario of a linear classifier and a statistical model that abstracts robust vs. non-robust features, the alternating best response strategy of such game may not converge. On the other hand, a unique pure Nash equilibrium of the game exists and is provably robust. We support our theoretical results with experiments, showing the non-convergence of adversarial train…
We investigate a model for representing large multiplayer games, which satisfy strong symmetry properties. This model is made of multiple copies of an arena; each player plays in his own arena, and can partially observe what the other players do. Therefore, this game has partial information and symmetry constraints, which make the computation of Nash equilibria difficult. We show several undecidability results, and for bounded-memory strategies, we precisely characterize the complexity of computing pure Nash equilibria (for qualitative objectives) in this game model.
We study the computational complexity of Nash equilibria in concurrent games with limit-average objectives. In particular, we prove that the existence of a Nash equilibrium in randomised strategies is undecidable, while the existence of a Nash equilibrium in pure strategies is decidable, even if we put a constraint on the payoff of the equilibrium. Our undecidability result holds even for a restricted class of concurrent games, where nonzero rewards occur only on terminal states. Moreover, we show that the constrained existence problem is undecidable not only for concurrent games but for turn-…
Our paper concerns the computation of Nash equilibria of first-price auctions with correlated values. While there exist several equilibrium computation methods for auctions with independent values, the correlation of the bidders' values introduces significant complications that render existing methods unsatisfactory in practice. Our contribution is a step towards filling this gap: inspired by the seminal fictitious play process of Brown and Robinson, we present a learning heuristic-that we call fictitious bidding (FB)-for estimating Bayes-Nash equilibria of first-price auctions with correlated…
We study the complexity of computing a uniform Nash equilibrium on a non-win-lose bimatrix game. It is known that such a problem is NP-complete even if a bimatrix game is win-lose (Bonifaci et al., 2008). Fortunately, if a win-lose bimatrix game is planar, then uniform Nash equilibria always exist. We have a polynomial-time algorithm for finding a uniform Nash equilibrium of a planar win-lose bimatrix game (Addario-Berry et al., 2007). The following question is left: How hard to compute a uniform Nash equilibrium on a planar non-win-lose bimatrix game? This paper resolves this issue. We prove …
We study the problem of checking for the existence of constrained pure Nash equilibria in a subclass of polymatrix games defined on weighted directed graphs. The payoff of a player is defined as the sum of nonnegative rational weights on incoming edges from players who picked the same strategy augmented by a fixed integer bonus for picking a given strategy. These games capture the idea of coordination within a local neighbourhood in the absence of globally common strategies. We study the decision problem of checking whether a given set of strategy choices for a subset of the players is consist…
This paper concerns computing approximate pure Nash equilibria in weighted congestion games, which has been shown to be PLS-complete. With the help of $\hatΨ$-game and approximate potential functions, we propose two algorithms based on best response dynamics, and prove that they efficiently compute $\fracρ{1-ε}$-approximate pure Nash equilibria for $ρ= d!$ and $ρ=\frac{2\cdot W\cdot(d+1)}{2\cdot W+d+1}\le {d + 1}$, respectively, when the weighted congestion game has polynomial latency functions of degree at most $d \ge 1$ and players' weights are bounded from above by a constant $W \ge 1$. Thi…
We prove that computing a Nash equilibrium of a two-player ($n \times n$) game with payoffs in $[-1,1]$ is PPAD-hard (under randomized reductions) even in the smoothed analysis setting, smoothing with noise of constant magnitude. This gives a strong negative answer to conjectures of Spielman and Teng [ST06] and Cheng, Deng, and Teng [CDT09]. In contrast to prior work proving PPAD-hardness after smoothing by noise of magnitude $1/\operatorname{poly}(n)$ [CDT09], our smoothed complexity result is not proved via hardness of approximation for Nash equilibria. This is by necessity, since Nash equil…
