Pavlov, a well-known strategy in game theory, has been shown to have some advantages in the Iterated Prisoner's Dilemma (IPD) game. However, this strategy can be exploited by inveterate defectors. We modify this strategy to mitigate the exploitation. We call the resulting strategy Rational Pavlov. This has a parameter p which measures the "degree of forgiveness" of the players. We study the evolution of cooperation in the IPD game, when n players are arranged in a cycle, and all play this strategy. We examine the effect of varying p on the convergence rate and prove that the convergence rate is fast, O(n log n) time, for high values of p. We also prove that the convergence rate is exponentially slow in n for small enough p. Our analysis leaves a gap in the range of p, but simulations suggest that there is, in fact, a sharp phase transition.
Prisoner's dilemma has been heavily studied. In classical model, each player chooses to either "Cooperate" or "Defect". In this paper, we generalize the prisoner's dilemma with a new alternative which is neither defect or cooperation. The classical model is the special case under the condition that the third state is not taken into consideration.
The Prisoner's Dilemma is a simple model that captures the essential contradiction between individual rationality and global rationality. Although the one-shot Prisoner's Dilemma is usually viewed simple, in this paper we will categorize it into five different types. For the type-4 Prisoner's Dilemma game, we will propose a self-enforcing algorithmic model to help non-cooperative agents obtain Pareto-efficient payoffs. The algorithmic model is based on an algorithm using complex numbers and can work in macro applications.
Changes in payoffs can transform Prisoner's Dilemma and other social dilemmas into harmonious win-win games. Using the Robinson-Goforth topology of 2x2 games, this paper analyzes how payoff swaps turn Prisoner's Dilemma into other games, compares Prisoner's Dilemmas with other families of games, traces paths that affect the difficulty of transforming Prisoner's Dilemma and other social dilemmas into win-win games, and shows how ties connect simpler and more complex games. Charts illustrate the relationships between the 144 strict ordinal 2x2 games, the 38 symmetric 2x2 ordinal games with and w…
We develop an efficient algorithm to determine the memory-depth of finite state machines and apply the algorithm to a collection of iterated prisoner's dilemma strategies. The calculation agrees with the memory-depth of other representations of common strategies such as Tit-For-Tat, Tit-For-2-Tats, etc. which are typically represented by lookup tables. Our algorithm allows the complexity of finite state machine based strategies to be characterized on the same footing as memory-n strategies.
We present tournament results and several powerful strategies for the Iterated Prisoner's Dilemma created using reinforcement learning techniques (evolutionary and particle swarm algorithms). These strategies are trained to perform well against a corpus of over 170 distinct opponents, including many well-known and classic strategies. All the trained strategies win standard tournaments against the total collection of other opponents. The trained strategies and one particular human made designed strategy are the top performers in noisy tournaments also.
Iterated Prisoner's Dilemma(IPD) is a well-known benchmark for studying the long term behaviors of rational agents, such as how cooperation can emerge among selfish and unrelated agents that need to co-exist over long term. Many well-known strategies have been studied, from the simple tit-for-tat(TFT) strategy made famous by Axelrod after his influential tournaments to more involved ones like zero determinant strategies studied recently by Press and Dyson. In this paper, following Press and Dyson, we consider one memory probabilistic strategies. We consider that we call invincible strategies: …
Self-Play Q-learners Can Provably Collude in the Iterated Prisoner's Dilemma
Université Jean Monnet Saint-Etienne（圣艾蒂安让莫内大学）；CNRS（法国国家科学研究中心）；Institut d’Optique Graduate School（高等光学学院）；Inria（法国国家信息与自动化研究所）；Laboratoire Hubert Curien UMR 5516（于贝尔·居里实验室）；Mila（米拉人工智能与智能系统研究所）；Université de Montréal（蒙特利尔大学）；Università LUISS (Rome)（罗马自由社会科学大学）；Toulouse School of Economics（图卢兹经济学院）；EIEF（意大利经济基金会）；CEPR（欧洲政策研究中心）；Canada AI CIFAR Chair（加拿大AI CIFA讲席）
Under the assumption of complete rationality, Nash equilibrium is the only reasonable strategy (set) of the finitely repeated prisoner's dilemma. In fact, some strategies only slightly deviate from the so-called rationality, and the corresponding payoff may much better than that of Nash equilibrium. This article points out, even under the rational assumptions, the players have reason to seek a mutually beneficial agreement (Pareto dominated compare to Nash equilibrium) and a weak and optional constraints, so that the agreement can be successfully implemented. If the constraint does not harm th…
How cooperation emerges and persists in a population of selfish agents is a fundamental question in evolutionary game theory. Our research shows that Collective Strategies with Master-Slave Mechanism (CSMSM) defeat Tit-for-Tat and other well-known strategies in spatial iterated prisoner's dilemma. A CSMSM identifies kin members by means of a handshaking mechanism. If the opponent is identified as non-kin, a CSMSM will always defect. Once two CSMSMs meet, they play master and slave roles. A mater defects and a slave cooperates in order to maximize the master's payoff. CSMSM outperforms non-coll…
The iterated prisoner's dilemma is a game that produces many counter-intuitive and complex behaviors in a social environment, based on very simple basic rules. It illustrates that cooperation can be a good thing even in a competitive world, that individual fitness needs not to be the most important criteria of success, and that some strategies are very strong in a direct confrontation but could still perform poorly on average or are evolutionarily unstable. In this contribution, we present a strategy -- PREDICTOR -- which appears to be "sentient" and chooses to cooperate when playing against s…
We explore some strategies which tend to perform well in the IPD. We start off by showing the significance of Tit-For-Tat strategies in evolutionary game theory. This is followed by a theoretical derivation of zero-determinant strategies, where we highlight an error on bounds for scale parameters from the original paper on ZD strategies[6]. We then present examples of such strategies and create a custom player drawing inspiration from Markov Decision Processes. At the end we pit them all against each other and see how they perform in an IPD tournament.
We study a program game version of the Prisoner's Dilemma, i.e., a two-player game in which each player submits a computer program, the programs are given read access to each other's source code and then choose whether to cooperate or defect. Prior work has introduced various programs that form cooperative equilibria against themselves in this game. For example, the $ε$-grounded Fair Bot cooperates with probability $ε$ and with the remaining probability runs its opponent's program and copies its action. If both players submit this program, then this is a Nash equilibrium in which both players …
We submitted two kinds of strategies to the iterated prisoner's dilemma (IPD) competitions organized by Graham Kendall, Paul Darwen and Xin Yao in 2004 and 2005. Our strategies performed exceedingly well in both years. One type is an intelligent and optimistic enhanced version of the well known TitForTat strategy which we named OmegaTitForTat. It recognizes common behaviour patterns and detects and recovers from repairable mutual defect deadlock situations, otherwise behaving much like TitForTat. The second type consists of a set of strategies working together as a team. These group strategies…
Strategic decision making involves affective and cognitive functions like reasoning, cognitive and emotional empathy which may be subject to age and gender differences. However, empathy-related changes in strategic decision-making and their relation to age, gender and neuropsychological functions have not been studied widely. In this article, we study a one-shot prisoner dilemma from a psychological game theory viewpoint. Forty seven participants (28 women and 19 men), aged 18 to 42 years, were tested with a empathy questionnaire and a one-shot prisoner dilemma questionnaire comprising a closi…
The Prisoners' Dilemma is perhaps the most famous model in the field of game theory. Consequently, it is natural to investigate its quantum version when one considers to apply quantum strategies to game theory. There are two main results in this paper: 1) The well-known Prisoners' Dilemma can be categorized into three types and only the third type is adaptable for quantum strategies. 2) As a reverse problem of game theory, mechanism design provides a better circumstance for quantum strategies than game theory does.
With deeper study of the Game Theory, some conditions of Prisoner's Dilemma is no longer suitable of games in real life. So we try to develop a new model-Villager's Dilemma which has more realistic conditions to stimulate the process of game. It is emphasize that Prisoner's Dilemma is an exception which is lack of universality and the importance of rules in the game. And it puts forward that to let the rule maker take part in the game and specifies game players can stop the game as they like. This essay describes the basic model, the villager's dilemma (VD) and put some extended use of it, and…
We investigate the repeated prisoner's dilemma game where both players alternately use reinforcement learning to obtain their optimal memory-one strategies. We theoretically solve the simultaneous Bellman optimality equations of reinforcement learning. We find that the Win-stay Lose-shift strategy, the Grim strategy, and the strategy which always defects can form symmetric equilibrium of the mutual reinforcement learning process amongst all deterministic memory-one strategies.
The 2 x 2 games, in particular the Prisoner's Dilemma, have been extensively used in studies into reciprocal cooperation and, to a lesser extent, kin selection. This paper examines the suitability of the 2 x 2 games for modelling the evolution of cooperation through reciprocation and kin selection. This examination is not restricted to the Prisoner's Dilemma, but includes the other non-trivial symmetric 2 x 2 games. We show that the popularity of the Prisoner's Dilemma for modelling social and biotic interaction is justified by its superiority according to these criteria. Indeed, the Prisoner'…
This paper examines the integration of computational complexity into game theoretic models. The example focused on is the Prisoner's Dilemma, repeated for a finite length of time. We show that a minimal bound on the players' computational ability is sufficient to enable cooperative behavior. In addition, a variant of the repeated Prisoner's Dilemma game is suggested, in which players have the choice of opting out. This modification enriches the game and suggests dominance of cooperative strategies. Competitive analysis is suggested as a tool for investigating sub-optimal (but computationally t…
We investigate symmetric equilibria of mutual reinforcement learning when both players alternately learn the optimal memory-two strategies against the opponent in the repeated prisoners' dilemma game. We provide a necessary condition for memory-two deterministic strategies to form symmetric equilibria. We then provide three examples of memory-two deterministic strategies which form symmetric mutual reinforcement learning equilibria. We also prove that mutual reinforcement learning equilibria formed by memory-two strategies are also mutual reinforcement learning equilibria when both players use…
Two-player games have had a long and fruitful history of applications stretching across the social, biological, and physical sciences. Most applications of two-player games assume synchronous decisions or moves even when the games are iterated. But different strategies may emerge as preferred when the decisions or moves are sequential, or the games are iterated. Zero-determinant strategies developed by Press and Dyson are a new class of strategies that have been developed for synchronous two-player games, most notably the iterated prisoner's dilemma. Here we apply the Press-Dyson analysis to s…
We consider the one-shot Prisoner's Dilemma between algorithms with read-access to one anothers' source codes, and we use the modal logic of provability to build agents that can achieve mutual cooperation in a manner that is robust, in that cooperation does not require exact equality of the agents' source code, and unexploitable, meaning that such an agent never cooperates when its opponent defects. We construct a general framework for such "modal agents", and study their properties.
We use replicator dynamics to study an iterated prisoners' dilemma game with memory. In this study, we investigate the characteristics of all 32 possible strategies with a single-step memory by observing the results when each strategy encounters another one. Based on these results, we define similarity measures between the 32 strategies and perform a network analysis of the relationship between the strategies by constructing a strategies network. Interestingly, we find that a win-lose circulation, like rock-paper-scissors, exists between strategies and that the circulation results from one unu…
For the iterated Prisoner's Dilemma, there exist Markov strategies which solve the problem when we restrict attention to the long term average payoff. When used by both players these assure the cooperative payoff for each of them. Neither player can benefit by moving unilaterally any other strategy, i.e. these are Nash equilibria. In addition, if a player uses instead an alternative which decreases the opponent's payoff below the cooperative level, then his own payoff is decreased as well. Thus, if we limit attention to the long term payoff, these \emph{good strategies} effectively stabilize c…
We analyze an extended model of the Iterated Prisoner's Dilemma where agents decide to play based on the data from their limited memory or recommendations. The cooperators can decide whether to play with the matched opponent or not. The agents' decisions are directly linked to their level of optimism since they decide to play if they believe the opponent has a high probability of cooperating. Optimism is precisely tuned by parameters optimism threshold and tolerance. Our experiment showed that being optimistic is better for cooperators as it leads to more accurate exploration in the multi-agen…
How attention scarcity effects the outcomes of a game? We present our findings on a version of the Iterated Prisoners Dilemma (IPD) game in which players can accept or refuse to play with their partner. We study the memory size effect on determining the right partner to interact with. We investigate the conditions under which the cooperators are more likely to be advantageous than the defectors. This work demonstrates that, in order to beat defection, players do not need a full memorization of each action of all opponents. There exists a critical attention capacity threshold to beat defectors.…
The Axelrod library is an open source Python package that allows for reproducible game theoretic research into the Iterated Prisoner's Dilemma. This area of research began in the 1980s but suffers from a lack of documentation and test code. The goal of the library is to provide such a resource, with facilities for the design of new strategies and interactions between them, as well as conducting tournaments and ecological simulations for populations of strategies. With a growing collection of 139 strategies, the library is a also a platform for an original tournament that, in itself, is of inte…
In this paper the results of a simulation of a prisoner's dilemma robin-round tournament are presented. In the tournament each participating strategy plays an iterated prisoner's dilemma against each other strategy (round-robin) and as a variant also against itself. The participants of a tournament are all strategies that are deterministic and have the same size of memory with regard to their own and their opponent's past actions: up to three most recent actions of their opponent and up to two most recent actions of their own. A focus is set on the investigation of the influence of the number …
In many cases the Nash equilibria are not predictive of the experimental players' behaviour. For some games of Game Theory it is proposed here a method to estimate the probabilities with which the different options will be actually chosen by balanced players, i.e. players that are neither too competitive, nor too cooperative. This will allow to measure the intrinsec cooperativeness degree of a game, only in function of its payoffs. The method is shaped on the Prisoner's Dilemma, then generalized for asymmetric tables, N players and N options. It is adapted to other conditions like Chicken Game…
Cooperation lies at the foundations of human societies, yet why people cooperate remains a conundrum. The issue, known as network reciprocity, of whether population structure can foster cooperative behavior in social dilemmas has been addressed by many, but theoretical studies have yielded contradictory results so far---as the problem is very sensitive to how players adapt their strategy. However, recent experiments with the prisoner's dilemma game played on different networks have shown that humans do not consider neighbors' payoffs when making their decisions, and that the network structure …
This paper studies scenarios of cyclic dominance in a coevolutionary spatial model in which game strategies and links between agents adaptively evolve over time. The Optional Prisoner's Dilemma (OPD) game is employed. The OPD is an extended version of the traditional Prisoner's Dilemma where players have a third option to abstain from playing the game. We adopt an agent-based simulation approach and use Monte Carlo methods to perform the OPD with coevolutionary rules. The necessary conditions to break the scenarios of cyclic dominance are also investigated. This work highlights that cyclic dom…
Tianjin University（天津大学）；Washington State University（华盛顿州立大学）
The Iterated Prisoner's Dilemma has guided research on social dilemmas for decades. However, it distinguishes between only two atomic actions: cooperate and defect. In real-world prisoner's dilemmas, these choices are temporally extended and different strategies may correspond to sequences of actions, reflecting grades of cooperation. We introduce a Sequential Prisoner's Dilemma (SPD) game to better capture the aforementioned characteristics. In this work, we propose a deep multiagent reinforcement learning approach that investigates the evolution of mutual cooperation in SPD games. Our approa…
We present insights and empirical results from an extensive numerical study of the evolutionary dynamics of the iterated prisoner's dilemma. Fixation probabilities for Moran processes are obtained for all pairs of 164 different strategies including classics such as TitForTat, zero determinant strategies, and many more sophisticated strategies. Players with long memories and sophisticated behaviours outperform many strategies that perform well in a two player setting. Moreover we introduce several strategies trained with evolutionary algorithms to excel at the Moran process. These strategies ar…
We study a model for switching strategies in the Prisoner's Dilemma game on adaptive networks of player pairings that coevolve as players attempt to maximize their return. We use a node-based strategy model wherein each player follows one strategy at a time (cooperate or defect) across all of its neighbors, changing that strategy and possibly changing partners in response to local changes in the network of player pairing and in the strategies used by connected partners. We compare and contrast numerical simulations with existing pair approximation differential equations for describing this sys…
High-frequency trading (HFT) offers an excellent user case and a potential killer application of the commercially available, first generation quasi-quantum communication and computation technologies. To this end, we offer here a simple but complete game-theoretic model of HFT as the famous two player game, Prisoner's Dilemma. We explore the implementation of HFT as a game on the (quasi) quantum cloud using the Eisert, Wilkens, and Lewenstein quantum mediated communication protocol, and how this implementation can increase transaction speed and improve the lot of the players in HFT. Using coope…
Research has shown that the addition of abstention as an option transforms social dilemmas to rock-paper-scissor type games, where defectors dominate cooperators, cooperators dominate abstainers (loners), and abstainers (loners), in turn, dominate defectors. In this way, abstention can sustain cooperation even under adverse conditions, although defection also persists due to cyclic dominance. However, to abstain or to act as a loner has, to date, always been considered as an independent, third strategy to complement traditional cooperation and defection. Here we consider probabilistic abstenti…
As an important psychological and social experiment, the Iterated Prisoner's Dilemma (IPD) treats the choice to cooperate or defect as an atomic action. We propose to study the behaviors of online learning algorithms in the Iterated Prisoner's Dilemma (IPD) game, where we investigate the full spectrum of reinforcement learning agents: multi-armed bandits, contextual bandits and reinforcement learning. We evaluate them based on a tournament of iterated prisoner's dilemma where multiple agents can compete in a sequential fashion. This allows us to analyze the dynamics of policies learned by mult…
Cooperative behavior lies at the very basis of human societies, yet its evolutionary origin remains a key unsolved puzzle. Whereas reciprocity or conditional cooperation is one of the most prominent mechanisms proposed to explain the emergence of cooperation in social dilemmas, recent experimental findings on networked Prisoner's Dilemma games suggest that conditional cooperation also depends on the previous action of the player---namely on the `mood' in which the player currently is. Roughly, a majority of people behaves as conditional cooperators if they cooperated in the past, while they ig…
We investigate the spatial distribution and the global frequency of agents who can either cooperate or defect. The agent interaction is described by a deterministic, non-iterated prisoner's dilemma game, further each agent only locally interacts with his neighbors. Based on a detailed analysis of the local payoff structures we derive critical conditions for the invasion or the spatial coexistence of cooperators and defectors. These results are concluded in a phase diagram that allows to identify five regimes, each characterized by a distinct spatiotemporal dynamics and a corresponding final sp…
We study a spatial, one-shot prisoner's dilemma (PD) model in which selection operates on both an organism's behavioral strategy (cooperate or defect) and its choice of when to implement that strategy across a set of discrete time slots. Cooperators evolve to fixation regularly in the model when we add time slots to lattices and small-world networks, and their portion of the population grows, albeit slowly, when organisms interact in a scale-free network. This selection for cooperators occurs across a wide variety of time slots and it does so even when a crucial condition for the evolution of …
Cooperation in one-shot anonymous interactions is a widely documented aspect of human behaviour. Here we shed light on the motivations behind this behaviour by experimentally exploring cooperation in a one-shot continuous-strategy Prisoner's Dilemma (i.e. one-shot two-player Public Goods Game). We examine the distribution of cooperation amounts, and how that distribution varies based on the benefit-to-cost ratio of cooperation (b/c). Interestingly, we find a trimodal distribution at all b/c values investigated. Increasing b/c decreases the fraction of participants engaging in zero cooperation …
This paper presents research comparing the effects of different environments on the outcome of an extended Prisoner's Dilemma, in which agents have the option to abstain from playing the game. We consider three different pure strategies: cooperation, defection and abstinence. We adopt an evolutionary game theoretic approach and consider two different environments: the first which imposes no spatial constraints and the second in which agents are placed on a lattice grid. We analyse the performance of the three strategies as we vary the loner's payoff in both structured and unstructured environm…
We have studied the impact of time-dependent learning capacities of players in the framework of spatial prisoner's dilemma game. In our model, this capacity of players may decrease or increase in time after strategy adoption according to a step-like function. We investigated both possibilities separately and observed significantly different mechanisms that form the stationary pattern of the system. The time decreasing learning activity helps cooperator domains to recover the possible intrude of defectors hence supports cooperation. In the other case the temporary restrained learning activity g…
The prisoner's dilemma game is the most known contribution of game theory into social sciences. Here we describe new implications of this game for transactional and transformative leadership. While the autocratic (Stackelberg's) leadership is inefficient for this game, we discuss a Pareto-optimal scenario, where the leader L commits to react probabilistically to pure strategies of the follower F, which is free to make the first move. Offering F to resolve the dilemma, L is able to get a larger average pay-off. The exploitation can be stabilized via repeated interaction of L and F, and turns to…
University of Electronic Science and Technology of China（电子科技大学）；Shanghai Jiao Tong University（上海交通大学）
Repeated game has long been the touchstone model for agents' long-run relationships. Previous results suggest that it is particularly difficult for a repeated game player to exert an autocratic control on the payoffs since they are jointly determined by all participants. This work discovers that the scale of a player's capability to unilaterally influence the payoffs may have been much underestimated. Under the conventional iterated prisoner's dilemma, we develop a general framework for controlling the feasible region where the players' payoff pairs lie. A control strategy player is able to co…
