Repeated games have provided an explanation how mutual cooperation can be achieved even if defection is more favorable in a one-shot game in prisoner's dilemma situation. Recently found zero-determinant strategies have substantially been investigated in evolutionary game theory. The original memory-one zero-determinant strategies unilaterally enforce linear relations between average payoffs of players. Here, we extend the concept of zero-determinant strategies to memory-two strategies in repeated games. Memory-two zero-determinant strategies unilaterally enforce linear relations between correlation functions of payoffs and payoffs at the previous round. Examples of memory-two zero-determinant strategy in the repeated prisoner's dilemma game are provided, some of which generalize the Tit-for-Tat strategy to memory-two case. Extension of zero-determinant strategies to memory-$n$ case with $n\geq 2$ is also straightforward.
We study the emergence of locally suboptimal behavior in finitely repeated games. Locally suboptimal behavior refers to players play suboptimally in some rounds of the repeated game (i.e., not maximizing their payoffs in those rounds) while maximizing their total payoffs in the whole repeated game. The central research question we aim to answer is when locally suboptimal behavior can arise from rational play in finitely repeated games. In this research, we focus on the emergence of locally suboptimal behavior in subgame-perfect equilibria (SPE) of finitely repeated games with complete informat…
Long-term cooperation, competition, or exploitation among individuals can be modeled through repeated games. In repeated games, Press and Dyson discovered zero-determinant (ZD) strategies that enforce a special relationship between two players. This special relationship implies that a ZD player can unilaterally impose a linear payoff relationship to the opponent regardless of the opponent's strategies. A ZD player also has a property that can lead the opponent to an unconditional cooperation if the opponent tries to improve its payoff. This property has been mathematically confirmed by Chen an…
Zero-determinant strategies are a class of memory-one strategies in repeated games which unilaterally enforce linear relationships between payoffs. It has long been unclear for what stage games zero-determinant strategies exist. We provide a necessary and sufficient condition for the existence of zero-determinant strategies. This condition can be interpreted as the existence of two different actions which unilaterally adjust the total value of a linear combination of payoffs. A relation between the class of stage games where zero-determinant strategies exist and other class of stage games is a…
Repeated games are difficult to analyze, especially when agents play mixed strategies. We study one-memory strategies in iterated prisoner's dilemma, then generalize the result to k-memory strategies in repeated games. Our result shows that there always exists a pure strategy best response, which can be computed with SMT or MDP solvers. However, there may not exist such pure strategy best response in multi-agent tournaments. All source code is released for verification.
Positive and Negative Determinant Strategies in Repeated Games with Behavior-Value Inconsistency
School of Mathematical Sciences, Beijing University of Posts and Telecommunications（北京邮电大学数学科学学院）；Key Laboratory of Mathematics and Information Networks (Beijing University of Posts and Telecommunications), Ministry of Education（教育部数学与信息网络重点实验室（北京邮电大学））；Department of Theoretical Biology, Max Planck Institute for Evolutionary Biology（马克斯·普朗克进化生物学研究所理论生物学系）；Department of Engineering, Universitat Pompeu Fabra（庞培法布拉大学工程学院）
In two-player repeated games, Zero-Determinant (ZD) strategies enable a player to unilaterally enforce a linear payoff relation between her own and her opponent's payoff irrespective of the opponent's strategy. This manipulative nature of the ZD strategies attracted significant attention from researchers due to its close connection to controlling distributively the outcome of evolutionary games in large populations. In this paper, necessary and sufficient conditions are derived for a payoff relation to be enforceable in multiplayer social dilemmas with a finite expected number of rounds that i…
This paper considers repeated games in which one player has more information about the game than the other players. In particular, we investigate repeated two-player zero-sum games where only the column player knows the payoff matrix A of the game. Suppose that while repeatedly playing this game, the row player chooses her strategy at each round by using a no-regret algorithm to minimize her (pseudo) regret. We develop a no-instant-regret algorithm for the column player to exhibit last round convergence to a minimax equilibrium. We show that our algorithm is efficient against a large set of po…
The central result of classical game theory states that every finite normal form game has a Nash equilibrium, provided that players are allowed to use randomized (mixed) strategies. However, in practice, humans are known to be bad at generating random-like sequences, and true random bits may be unavailable. Even if the players have access to enough random bits for a single instance of the game their randomness might be insufficient if the game is played many times. In this work, we ask whether randomness is necessary for equilibria to exist in finitely repeated games. We show that for a large …
We suggest a general method for inferring players' values from their actions in repeated games. The method extends and improves upon the recent suggestion of (Nekipelov et al., EC 2015) and is based on the assumption that players are more likely to exhibit sequences of actions that have lower regret. We evaluate this "quantal regret" method on two different datasets from experiments of repeated games with controlled player values: those of (Selten and Chmura, AER 2008) on a variety of two-player 2x2 games and our own experiment on ad-auctions (Noti et al., WWW 2014). We find that the quantal r…
One of the proposed solutions to the equilibrium selection problem for agents learning in repeated games is obtained via the notion of stochastic stability. Learning algorithms are perturbed so that the Markov chain underlying the learning dynamics is necessarily irreducible and yields a unique stable distribution. The stochastically stable distribution is the limit of these stable distributions as the perturbation rate tends to zero. We present the first exact algorithm for computing the stochastically stable distribution of a Markov chain. We use our algorithm to predict the long-term dynami…
We present a scheme for playing quantum repeated 2x2 games based on the Marinatto and Weber's approach to quantum games. As a potential application, we study twice repeated Prisoner's Dilemma game. We show that results not available in classical game can be obtained when the game is played in the quantum way. Before we present our idea, we comment on the previous scheme of playing quantum repeated games.
We present a polynomial-time algorithm that always finds an (approximate) Nash equilibrium for repeated two-player stochastic games. The algorithm exploits the folk theorem to derive a strategy profile that forms an equilibrium by buttressing mutually beneficial behavior with threats, where possible. One component of our algorithm efficiently searches for an approximation of the egalitarian point, the fairest pareto-efficient solution. The paper concludes by applying the algorithm to a set of grid games to illustrate typical solutions the algorithm finds. These solutions compare very favorably…
This paper presents a technique for approximating, up to any precision, the set of subgame-perfect equilibria (SPE) in discounted repeated games. The process starts with a single hypercube approximation of the set of SPE. Then the initial hypercube is gradually partitioned on to a set of smaller adjacent hypercubes, while those hypercubes that cannot contain any point belonging to the set of SPE are simultaneously withdrawn. Whether a given hypercube can contain an equilibrium point is verified by an appropriate mathematical program. Three different formulations of the algorithm for both appro…
Zero-sum asymmetric games model decision making scenarios involving two competing players who have different information about the game being played. A particular case is that of nested information, where one (informed) player has superior information over the other (uninformed) player. This paper considers the case of nested information in repeated zero-sum games and studies the computation of strategies for both the informed and uninformed players for finite-horizon and discounted infinite-horizon nested information games. For finite-horizon settings, we exploit that for both players, the se…
We prove that every repeated game with countably many players, finite action sets, and tail-measurable payoffs admits an $ε$-equilibrium, for every $ε> 0$.
A repeated game is an effective tool to model interactions and conflicts for players aiming to achieve their objectives in a long-term basis. Contrary to static noncooperative games that model an interaction among players in only one period, in repeated games, interactions of players repeat for multiple periods; and thus the players become aware of other players' past behaviors and their future benefits, and will adapt their behavior accordingly. In wireless networks, conflicts among wireless nodes can lead to selfish behaviors, resulting in poor network performances and detrimental individual…
We study the quality of outcomes in repeated games when the population of players is dynamically changing and participants use learning algorithms to adapt to the changing environment. Game theory classically considers Nash equilibria of one-shot games, while in practice many games are played repeatedly, and in such games players often use algorithmic tools to learn to play in the given environment. Most previous work on learning in repeated games assumes that the population playing the game is static over time. We analyze the efficiency of repeated games in dynamically changing environments, …
We study infinitely repeated games in settings of imperfect monitoring. We first prove a family of theorems that show that when the signals observed by the players satisfy a condition known as $(ε, γ)$-differential privacy, that the folk theorem has little bite: for values of $ε$ and $γ$ sufficiently small, for a fixed discount factor, any equilibrium of the repeated game involve players playing approximate equilibria of the stage game in every period. Next, we argue that in large games ($n$ player games in which unilateral deviations by single players have only a small impact on the utility o…
We consider infinite duration alternating move games. These games were previously studied by Roth, Balcan, Kalai and Mansour. They presented an FPTAS for computing an approximated equilibrium, and conjectured that there is a polynomial algorithm for finding an exact equilibrium. We extend their study in two directions: (1) We show that finding an exact equilibrium, even for two-player zero-sum games, is polynomial time equivalent to finding a winning strategy for a (two-player) mean-payoff game on graphs. The existence of a polynomial algorithm for the latter is a long standing open question i…
In this paper, we introduce the concept of infinitely split Nash equilibrium in repeated games in which the profile sets are chain-complete posets. Then by using a fixed point theorem on posets in [8], we prove an existence theorem. As an application, we study the repeated extended Bertrant duopoly model of price competition.
In a stochastic Stackelberg asymmetric security game, the strong Stackelberg equilibrium (SSE) strategy is a popular option for the defender to get the highest utility against an attacker with the best response (BR) strategy. However, the attacker may be a boundedly rational player, who adopts a combination of the BR strategy and a fixed stubborn one. In such a condition, the SSE strategy may not maintain the defensive performance due to the stubborn element. In this paper, we focus on how the defender can adopt the unilateral-control zero-determinate (ZD) strategy to confront the boundedly ra…
It is well-known that for infinitely repeated games, there are computable strategies that have best responses, but no computable best responses. These results were originally proved for either specific games (e.g., Prisoner's dilemma), or for classes of games satisfying certain conditions not known to be both necessary and sufficient. We derive a complete characterization in the form of simple necessary and sufficient conditions for the existence of a computable strategy without a computable best response under limit-of-means payoff. We further refine the characterization by requiring the stra…
Chalmers University of Technology（查尔姆斯理工大学）；Microsoft Research Lab（微软研究院）
We study two-player general sum repeated finite games where the rewards of each player are generated from an unknown distribution. Our aim is to find the egalitarian bargaining solution (EBS) for the repeated game, which can lead to much higher rewards than the maximin value of both players. Our most important contribution is the derivation of an algorithm that achieves simultaneously, for both players, a high-probability regret bound of order $\mathcal{O}(\sqrt[3]{\ln T}\cdot T^{2/3})$ after any $T$ rounds of play. We demonstrate that our upper bound is nearly optimal by proving a lower bound…
Refundable Deposits: How to Restore Cooperation in Finitely Repeated Games
SYCAMORE Lab, EPFL - Swiss Federal Technology Institute of Lausanne（洛桑联邦理工学院 S Y C A M O R E 实验室）；Department of Mathematics, London School of Economics and Political Science（伦敦政治经济学院数学系）；Department of Quantitative Methods, CUNEF Universidad（CUNEF 大学定量方法系）
A formula is presented for designing zero-determinant(ZD) strategies of general finite games, which have $n$ players and players can have different numbers of strategies. To this end, using semi-tensor product (STP) of matrices, the profile evolutionary equation for repeated finite games is obtained. Starting from it, the ZD strategies are developed for general finite games, based on the same technique proposed by Press and Dyson \cite{pre12}. A formula is obtain to design ZD strategies for any player directly, ignoring the original ZD design process. Necessary and sufficient condition is obta…
This paper studies the mechanisms, implications, and potential applications of the recently discovered class of Zero Determinant (ZD) strategies in iterated 2x2 games. These strategies were reported to successfully extort pure economic maximizers, and to mischievously determine the set of feasible long-term payoffs in iterated Prisoners' Dilemma by enforcing linear constraints on both players' expected average scores. These results are generalized for all symmetric 2x2 games and a general Battle of the Sexes, exemplified by four common games. Additionally, a comparison to conventional strategi…
We present a method of backward induction for computing approximate subgame perfect Nash equilibria of infinitely repeated games with discounted payoffs. This uses the selection monad transformer, combined with the searchable set monad viewed as a notion of 'topologically compact' nondeterminism, and a simple model of computable real numbers. This is the first application of Escardó and Oliva's theory of higher-order sequential games to games of imperfect information, in which (as well as its mathematical elegance) lazy evaluation does nontrivial work for us compared with a traditional game-th…
Zero-determinant strategies are memory-one strategies in repeated games which unilaterally enforce linear relations between expected payoffs of players. Recently, the concept of zero-determinant strategies was extended to the class of memory-$n$ strategies with $n\geq 1$, which enables more complicated control of payoffs by one player. However, what we can do by memory-$n$ zero-determinant strategies is still not clear. Here, we show that memory-$n$ zero-determinant strategies in repeated games can be used to control conditional expectations of payoffs. Equivalently, they can be used to contro…
We investigate symmetric equilibria of mutual reinforcement learning when both players alternately learn the optimal memory-two strategies against the opponent in the repeated prisoners' dilemma game. We provide a necessary condition for memory-two deterministic strategies to form symmetric equilibria. We then provide three examples of memory-two deterministic strategies which form symmetric mutual reinforcement learning equilibria. We also prove that mutual reinforcement learning equilibria formed by memory-two strategies are also mutual reinforcement learning equilibria when both players use…
In communication systems where users share common resources, users' selfish behavior usually results in suboptimal resource utilization. There have been extensive works that model communication systems with selfish users as one-shot games and propose incentive schemes to achieve Pareto optimal action profiles as non-cooperative equilibria. However, in many communication systems, due to strong negative externalities among users, the sets of feasible payoffs in one-shot games are nonconvex. Thus, it is possible to expand the set of feasible payoffs by having users choose convex combinations of d…
This paper studies two-player zero-sum repeated Bayesian games in which every player has a private type that is unknown to the other player, and the initial probability of the type of every player is publicly known. The types of players are independently chosen according to the initial probabilities, and are kept the same all through the game. At every stage, players simultaneously choose actions, and announce their actions publicly. For finite horizon cases, an explicit linear program is provided to compute players' security strategies. Moreover, based on the existing results in [1], this pap…
This paper considers a time-varying game with $N$ players. Every time slot, players observe their own random events and then take a control action. The events and control actions affect the individual utilities earned by each player. The goal is to maximize a concave function of time average utilities subject to equilibrium constraints. Specifically, participating players are provided access to a common source of randomness from which they can optimally correlate their decisions. The equilibrium constraints incentivize participation by ensuring that players cannot earn more utility if they cho…
We study the existence of different notions of value in two-person zero-sum repeated games where the state evolves and players receive signals. We provide some examples showing that the limsup value (and the uniform value) may not exist in general. Then we show the existence of the value for any Borel payoff function if the players observe a public signal including the actions played. We also prove two other positive results without assumptions on the signaling structure: the existence of the $\sup$ value in any game and the existence of the uniform value in recursive games with nonnegative pa…
We introduce the concept of attainable sets of payoffs in two-player repeated games with vector payoffs. A set of payoff vectors is called {\em attainable} if player 1 can ensure that there is a finite horizon $T$ such that after time $T$ the distance between the set and the cumulative payoff is arbitrarily small, regardless of what strategy player 2 is using. This paper focuses on the case where the attainable set consists of one payoff vector. In this case the vector is called an attainable vector. We study properties of the set of attainable vectors, and characterize when a specific vector …
This paper formally models the strategic repeated interactions between a system, comprising of a machine learning (ML) model and associated explanation method, and an end-user who is seeking a prediction/label and its explanation for a query/input, by means of game theory. In this game, a malicious end-user must strategically decide when to stop querying and attempt to compromise the system, while the system must strategically decide how much information (in the form of noisy explanations) it should share with the end-user and when to stop sharing, all without knowing the type (honest/maliciou…
Zero-determinant (ZD) strategies, a recently found novel class of strategies in repeated games, has attracted much attention in evolutionary game theory. A ZD strategy unilaterally enforces a linear relation between average payoffs of players. Although existence and evolutional stability of ZD strategies have been studied in simple games, their mathematical properties have not been well-known yet. For example, what happens when more than one players employ ZD strategies have not been clarified. In this paper, we provide a general framework for investigating situations where more than one playe…
This paper investigates repeated win-lose coordination games (WLC-games). We analyse which protocols are optimal for these games covering both the worst case and average case scenarios, i,e., optimizing the guaranteed and expected coordination times. We begin by analysing Choice Matching Games (CM-games) which are a simple yet fundamental type of WLC-games, where the goal of the players is to pick the same choice from a finite set of initially indistinguishable choices. We give a complete classification of optimal expected and guaranteed coordination times in two-player CM-games and show that …
In this note, we consider repeated play of a finite game using learning rules whose period-by-period behavior probabilities or empirical distributions converge to some notion of equilibria of the stage game. Our primary focus is on uncoupled and completely uncoupled learning rules. While the former relies on players being aware of only their own payoff functions and able to monitor the action taken by the others, the latter assumes that players only know their own past realized payoffs. We highlight the border between possible and impossible results using these rules. We also overview several …
Recently, Press and Dyson have proposed a new class of probabilistic and conditional strategies for the two-player iterated Prisoner's Dilemma, so-called zero-determinant strategies. A player adopting zero-determinant strategies is able to pin the expected payoff of the opponents or to enforce a linear relationship between his own payoff and the opponents' payoff, in a unilateral way. This paper considers zero-determinant strategies in the iterated public goods game, a representative multi-player evolutionary game where in each round each player will choose whether or not put his tokens into a…
Imperial College London（伦敦帝国学院）；University of Southampton（南安普顿大学）
Game theory's prescriptive power typically relies on full rationality and/or self-play interactions. In contrast, this work sets aside these fundamental premises and focuses instead on heterogeneous autonomous interactions between two or more agents. Specifically, we introduce a new and concise representation for repeated adversarial (constant-sum) games that highlight the necessary features that enable an automated planing agent to reason about how to score above the game's Nash equilibrium, when facing heterogeneous adversaries. To this end, we present TeamUP, a model-based RL algorithm desi…
In this article, we study the repeated routing game problem on a parallel network with affine latency functions on each edge. We cast the game setup in a LQR control theoretic framework, leveraging the Rosenthal potential formulation. We use control techniques to analyze the convergence of the game dynamics with specific cases that lend themselves to optimal control. We design proper dynamics parameters so that the conservation of flow is guaranteed. We provide an algorithmic solution for the general optimal control setup using a multiparametric quadratic programming approach (explicit MPC). F…
We consider repeated zero-sum games with incomplete information on the side of Player 2 with the total payoff given by the non-normalized sum of stage gains. In the classical examples the value $V_N$ of such an $N$-stage game is of the order of $N$ or $\sqrt{N}$ as $N\to \infty$. Our aim is to find what is causing another type of asymptotic behavior of the value $V_N$ observed for the discrete version of the financial market model introduced by De Meyer and Saley. For this game Domansky and independently De Meyer with Marino found that $V_N$ remains bounded as $N\to\infty$ and converges to the…
We consider eager-push epidemic dissemination in a complete graph. Time is divided into synchronous stages. In each stage, a source disseminates $ν$ events. Each event is sent by the source, and forwarded by each node upon its first reception, to $f$ nodes selected uniformly at random, where $f$ is the fanout. We use Game Theory to study the range of $f$ for which equilibria strategies exist, assuming that players are either rational or obedient to the protocol, and that they do not collude. We model interactions as an infinitely repeated game. We devise a monitoring mechanism that extends the…
Imitation is simple behavior which uses successful actions of others in order to deal with one's own problems. Because success of imitation generally depends on whether profit of an imitating agent coincides with those of other agents or not, game theory is suitable for specifying situations where imitation can be successful. One of the concepts describing successfulness of imitation in repeated two-player symmetric games is unbeatability. For infinitely repeated two-player symmetric games, a necessary and sufficient condition for some imitation strategy to be unbeatable was specified. However…
