We consider a dynamical approach to sequential games. By restricting the convertibility relation over strategy profiles, we obtain a semi-potential (in the sense of Kukushkin), and we show that in finite games the corresponding restriction of better-response dynamics will converge to a Nash equilibrium in quadratic time. Convergence happens on a per-player basis, and even in the presence of players with cyclic preferences, the players with acyclic preferences will stabilize. Thus, we obtain a candidate notion for rationality in the presence of irrational agents. Moreover, the restriction of convertibility can be justified by a conservative updating of beliefs about the other players strategies.
For infinite sequential games we can retain convergence to a Nash equilibrium (in some sense), if the preferences are given by continuous payoff functions; or obtain a transfinite convergence if the outcome sets of the game are Delta^0_2 sets.
We apply ideas from abstract argumentation theory to study cooperative game theory. Building on Dung's results in his seminal paper, we further the correspondence between Dung's four argumentation semantics and solution concepts in cooperative game theory by showing that complete extensions (the grounded extension) correspond to Roth's subsolutions (respectively, the supercore). We then investigate the relationship between well-founded argumentation frameworks and convex games, where in each case the semantics (respectively, solution concepts) coincide; we prove that three-player convex games …
Extensive-form games (EFGs) model finite sequential interactions between players. The amount of memory required to represent these games is the main bottleneck of algorithms for computing optimal strategies and the size of these strategies is often impractical for real-world applications. A common approach to tackle the memory bottleneck is to use information abstraction that removes parts of information available to players thus reducing the number of decision points in the game. However, existing information-abstraction techniques are either specific for a particular domain, they do not prov…
Supermodular games find significant applications in a variety of models, especially in operations research and economic applications of noncooperative game theory, and feature pure strategy Nash equilibria characterized as fixed points of multivalued functions on complete lattices. Pure strategy Nash equilibria of supermodular games are here approximated by resorting to the theory of abstract interpretation, a well established and known framework used for designing static analyses of programming languages. This is obtained by extending the theory of abstract interpretation in order to handle a…
Politecnico di Milano（米兰理工大学）；Politecnico di Torino（都灵理工大学）
\emph{Ex ante} correlation is becoming the mainstream approach for \emph{sequential adversarial team games}, where a team of players faces another team in a zero-sum game. It is known that team members' asymmetric information makes both equilibrium computation \textsf{APX}-hard and team's strategies not directly representable on the game tree. This latter issue prevents the adoption of successful tools for huge 2-player zero-sum games such as, \emph{e.g.}, abstractions, no-regret learning, and subgame solving. This work shows that we can recover from this weakness by bridging the gap between s…
We present a simple primal-dual algorithm for computing approximate Nash-equilibria in two-person zero-sum sequential games with incomplete information and perfect recall (like Texas Hold'em Poker). Our algorithm is numerically stable, performs only basic iterations (i.e matvec multiplications, clipping, etc., and no calls to external first-order oracles, no matrix inversions, etc.), and is applicable to a broad class of two-person zero-sum games including simultaneous games and sequential games with incomplete information and perfect recall. The applicability to the latter kind of games is th…
This paper analyses Escardó and Oliva's generalisation of selection functions over a strong monad from a game-theoretic perspective. We focus on the case of the nondeterminism (finite nonempty powerset) monad $\mathcal{P}$. We use these nondeterministic selection functions of type $\mathcal{J}^{\mathcal{P}}_R X = (X \rightarrow R) \rightarrow \mathcal{P} (X)$ to study sequential games, extending previous work linking (deterministic) selection functions to game theory. Similar to deterministic selection functions, which compute a subgame perfect Nash equilibrium play of a game, we characterise …
The travel costs of the players (travelers) in anonymous congestion games depend on their choices of routes and also on the states of the transportation network such as incidents, weather, and road work. In this extended abstract, we consider an incomplete-information environment in which the realizations of the states are unobserved by the travelers. We study how a planner can incentivize the travelers to behave in her favor by strategically designing what and how the travelers get informed about the realizations of the states.
We study the combinatorial two-player game Tron. We answer the extremal question on general graphs and also consider smaller graph classes. Bodlaender and Kloks conjectured in [2] PSPACE- completeness. We proof this conjecture.
Using Sperner's lemma for modified partition of a simplex we will constructively prove the existence of a Nash equilibrium in a finite strategic game with sequentially locally non-constant payoff functions. We follow the Bishop style constructive mathematics.
In this paper, we analyze a transportation game first introduced by Fotakis, Gourvès, and Monnot in 2017, where players want to be transported to a common destination as quickly as possible and, in order to achieve this goal, they have to choose one of the available buses. We introduce a sequential version of this game and provide bounds for the Sequential Price of Stability and the Sequential Price of Anarchy in both metric and non-metric instances, considering three social cost functions: the total traveled distance by all buses, the maximum distance traveled by a bus, and the sum of the dis…
We examine sequential equilibrium in the context of computational games, where agents are charged for computation. In such games, an agent can rationally choose to forget, so issues of imperfect recall arise. In this setting, we consider two notions of sequential equilibrium. One is an ex ante notion, where a player chooses his strategy before the game starts and is committed to it, but chooses it in such a way that it remains optimal even off the equilibrium path. The second is an interim notion, where a player can reconsider at each information set whether he is doing the "right" thing, and …
Czech Technical University in Prague（布拉格捷克理工大学）；Hebrew University of Jerusalem（耶路撒冷希伯来大学）；Aarhus University（奥胡斯大学）；IT-University of Copenhagen（哥本哈根信息技术大学）
The Stackelberg equilibrium solution concept describes optimal strategies to commit to: Player 1 (termed the leader) publicly commits to a strategy and Player 2 (termed the follower) plays a best response to this strategy (ties are broken in favor of the leader). We study Stackelberg equilibria in finite sequential games (or extensive-form games) and provide new exact algorithms, approximate algorithms, and hardness results for several classes of these sequential games.
Exploiting the algebraic structure of the set of bimatrix games, a divide-and-conquer algorithm for finding Nash equilibria is proposed. The algorithm is fixed-parameter tractable with the size of the largest irreducible component of a game as parameter. An implementation of the algorithm is shown to yield a significant performance increase on inputs with small parameters.
Stackelberg equilibrium is a solution concept that describes optimal strategies to commit: Player 1 (the leader) first commits to a strategy that is publicly announced, then Player 2 (the follower) plays a best response to the leader's commitment. We study the problem of computing Stackelberg equilibria in sequential games with finite and indefinite horizons, when players can play history-dependent strategies. Using the alternate formulation called strategies with memory, we establish that strategy profiles with polynomial memory size can be described efficiently. We prove that there exist a p…
Voting by sequential elimination is a low-communication voting protocol: voters play in sequence and eliminate one or more of the remaining candidates, until only one remains. While the fairness and efficiency of such protocols have been explored, the impact of strategic behaviour has not been addressed. We model voting by sequential elimination as a game. Given a fixed elimination sequence, we show that the outcome is the same in all subgame-perfect Nash equilibria of the corresponding game, and is polynomial-time computable. We measure the loss of social welfare due to strategic behaviour, w…
The paper studies properties of functional dependencies between strategies of players in Nash equilibria of multi-player strategic games. The main focus is on the properties of functional dependencies in the context of a fixed dependency graph for pay-off functions. A logical system describing properties of functional dependence for any given graph is proposed and is proven to be complete.
We consider the degrees of non-computability (Weihrauch degrees) of finding winning strategies (or more generally, Nash equilibria) in infinite sequential games with certain winning sets (or more generally, outcome sets). In particular, we show that as the complexity of the winning sets increases in the difference hierarchy, the complexity of constructing winning strategies increases in the effective Borel hierarchy.
The paper introduces a generic approach to solving Sequential Security Games (SGs) which utilizes Evolutionary Algorithms. Formulation of the method (named EASG) is general and largely game-independent, which allows for its application to a wide range of SGs with just little adjustments addressing game specificity. Comprehensive experiments performed on 3 different types of games (with 300 instances in total) demonstrate robustness and stability of EASG, manifested by repeatable achieving optimal or near-optimal solutions in the vast majority of the cases. The main advantage of EASG is time ef…
We study routing games where every agent sequentially decides her next edge when she obtains the green light at each vertex. Because every edge only has capacity to let out one agent per round, an edge acts as a FIFO waiting queue that causes congestion on agents who enter. Given $n$ agents over $|V|$ vertices, we show that for one agent, approximating a winning strategy within $n^{1-\varepsilon}$ of the optimum for any $\varepsilon>0$, or within any polynomial of $|V|$, are PSPACE-hard. Under perfect information, computing a subgame perfect equilibrium (SPE) is PSPACE-hard and in FPSPACE. Und…
We study pure Nash equilibria in games on graphs with an imperfect monitoring based on a public signal. In such games, deviations and players responsible for those deviations can be hard to detect and track. We propose a generic epistemic game abstraction, which conveniently allows to represent the knowledge of the players about these deviations, and give a characterization of Nash equilibria in terms of winning strategies in the abstraction. We then use the abstraction to develop algorithms for some payoff functions.
We consider N-player non-zero sum games played on finite trees (i.e., sequential games), in which the players have the right to repeatedly update their respective strategies (for instance, to improve the outcome wrt to the current strategy profile). This generates a dynamics in the game which may eventually stabilise to a Nash Equilibrium (as with Kukushkin's lazy improvement), and we argue that it is interesting to study the conditions that guarantee such a dynamics to terminate. We build on the works of Le Roux and Pauly who have studied extensively one such dynamics, namely the Lazy Improve…
Politecnico di Milano（米兰理工大学）；University of Southampton（南安普顿大学）
We investigate the computation of equilibria in extensive-form games where ex ante correlation is possible, focusing on correlated equilibria requiring the least amount of communication between the players and the mediator. Motivated by the hardness results on the computation of normal-form correlated equilibria, we introduce the notion of normal-form coarse correlated equilibrium, extending the definition of coarse correlated equilibrium to sequential games. We show that, in two-player games without chance moves, an optimal (e.g., social welfare maximizing) normal-form coarse correlated equil…
Despite the emphases on computability issues in research of algorithmic game theory, the limited computational capacity of players have received far less attention. This work examines how different levels of players' computational ability (or "rationality") impact the outcomes of sequential scheduling games. Surprisingly, our results show that a lower level of rationality of players may lead to better equilibria. More specifically, we characterize the sequential price of anarchy (SPoA) under two different models of bounded rationality, namely, players with $k$-lookahead and simple-minded playe…
We consider a dynamical approach to game in extensive forms. By restricting the convertibility relation over strategy profiles, we obtain a semi-potential (in the sense of Kukushkin), and we show that in finite games the corresponding restriction of better-response dynamics will converge to a Nash equilibrium in quadratic (finite) time. Convergence happens on a per-player basis, and even in the presence of players with cyclic preferences, the players with acyclic preferences will stabilize. Thus, we obtain a candidate notion for rationality in the presence of irrational agents. Moreover, the r…
Prior work on the complexity of bribery assumes that the bribery happens simultaneously, and that the briber has full knowledge of all voters' votes. But neither of those assumptions always holds. In many real-world settings, votes come in sequentially, and the briber may have a use-it-or-lose-it moment to decide whether to bribe/alter a given vote, and at the time of making that decision, the briber may not know what votes remaining voters are planning on casting. In this paper, we introduce a model for, and initiate the study of, bribery in such an online, sequential setting. We show that ev…
The peculiarity of adversarial team games resides in the asymmetric information available to the team members during the play, which makes the equilibrium computation problem hard even with zero-sum payoffs. The algorithms available in the literature work with implicit representations of the strategy space and mainly resort to Linear Programming and column generation techniques to enlarge incrementally the strategy space. Such representations prevent the adoption of standard tools such as abstraction generation, game solving, and subgame solving, which demonstrated to be crucial when solving h…
In the context of multi-player, general-sum games, there is an increasing interest in solution concepts modeling some form of communication among players, since they can lead to socially better outcomes with respect to Nash equilibria, and may be reached through learning dynamics in a decentralized fashion. In this paper, we focus on coarse correlated equilibria (CCEs) in sequential games. First, we complete the picture on the complexity of finding social-welfare-maximizing CCEs by showing that the problem is not in Poly-APX unless P = NP. Furthermore, simple arguments show that CFR - working …
We define solution concepts appropriate for computationally bounded players playing a fixed finite game. To do so, we need to define what it means for a \emph{computational game}, which is a sequence of games that get larger in some appropriate sense, to represent a single finite underlying extensive-form game. Roughly speaking, we require all the games in the sequence to have essentially the same structure as the underlying game, except that two histories that are indistinguishable (i.e., in the same information set) in the underlying game may correspond to histories that are only computation…
We investigate Kantian equilibria in finite normal form games, a class of non-Nashian, morally motivated courses of action that was recently proposed in the economics literature. We highlight a number of problems with such equilibria, including computational intractability, a high price of miscoordination, and problematic extension to general normal form games. We give such a generalization based on concept of program equilibria, and point out that that a practically relevant generalization may not exist. To remedy this we propose some general, intuitive, computationally tractable, other-regar…
We investigate the existence of certain types of equilibria (Nash, $\varepsilon$-Nash, subgame perfect, $\varepsilon$-subgame perfect, Pareto-optimal) in multi-player multi-outcome infinite sequential games. We use two fundamental approaches: one requires strong topological restrictions on the games, but produces very strong existence results. The other merely requires some very basic determinacy properties to still obtain some existence results. Both results are transfer results: starting with the existence of some equilibria for a small class of games, they allow us to conclude the existence…
Matrix games like Prisoner's Dilemma have guided research on social dilemmas for decades. However, they necessarily treat the choice to cooperate or defect as an atomic action. In real-world social dilemmas these choices are temporally extended. Cooperativeness is a property that applies to policies, not elementary actions. We introduce sequential social dilemmas that share the mixed incentive structure of matrix game social dilemmas but also require agents to learn policies that implement their strategic intentions. We analyze the dynamics of policies learned by multiple self-interested indep…
Media and Data Science Research Lab, Adobe（Adobe媒体与数据科学研究实验室）；IIT Kharagpur, India（印度理工学院卡格拉普尔分校）
In social dilemma situations, individual rationality leads to sub-optimal group outcomes. Several human engagements can be modeled as a sequential (multi-step) social dilemmas. However, in contrast to humans, Deep Reinforcement Learning agents trained to optimize individual rewards in sequential social dilemmas converge to selfish, mutually harmful behavior. We introduce a status-quo loss (SQLoss) that encourages an agent to stick to the status quo, rather than repeatedly changing its policy. We show how agents trained with SQLoss evolve cooperative behavior in several social dilemma matrix ga…
Two-player games have had a long and fruitful history of applications stretching across the social, biological, and physical sciences. Most applications of two-player games assume synchronous decisions or moves even when the games are iterated. But different strategies may emerge as preferred when the decisions or moves are sequential, or the games are iterated. Zero-determinant strategies developed by Press and Dyson are a new class of strategies that have been developed for synchronous two-player games, most notably the iterated prisoner's dilemma. Here we apply the Press-Dyson analysis to s…
We initiate the study of trembling-hand perfection in sequential (i.e., extensive-form) games with correlation. We introduce the extensive-form perfect correlated equilibrium (EFPCE) as a refinement of the classical extensive-form correlated equilibrium (EFCE) that amends its weaknesses off the equilibrium path. This is achieved by accounting for the possibility that players may make mistakes while following recommendations independently at each information set of the game. After providing an axiomatic definition of EFPCE, we show that one always exists since any perfect (Nash) equilibrium con…
Tianjin University（天津大学）；Washington State University（华盛顿州立大学）
The Iterated Prisoner's Dilemma has guided research on social dilemmas for decades. However, it distinguishes between only two atomic actions: cooperate and defect. In real-world prisoner's dilemmas, these choices are temporally extended and different strategies may correspond to sequences of actions, reflecting grades of cooperation. We introduce a Sequential Prisoner's Dilemma (SPD) game to better capture the aforementioned characteristics. In this work, we propose a deep multiagent reinforcement learning approach that investigates the evolution of mutual cooperation in SPD games. Our approa…
We study a decentralized matching market in which firms sequentially make offers to potential workers. For each offer, the worker can choose "accept" or "reject," but the decision is irrevocable. The acceptance of an offer guarantees her job at the firm, but it may also eliminate chances of better offers from other firms in the future. We formulate this market as a perfect-information extensive-form game played by the workers. Each instance of this game has a unique subgame perfect equilibrium (SPE), which does not necessarily lead to a stable matching and has some perplexing properties. We sh…
In this paper, we consider a discrete-time stochastic Stackelberg game with a single leader and multiple followers. Both the followers and the leader together have conditionally independent private types, conditioned on action and previous state, that evolve as controlled Markov processes. The objective is to compute the stochastic Stackelberg equilibrium of the game where the leader commits to a dynamic strategy. Each follower's strategy is the best response to the leader's strategies and other followers' strategies while the each leader's strategy is optimum given the followers play the best…
In this paper, we present exploitability descent, a new algorithm to compute approximate equilibria in two-player zero-sum extensive-form games with imperfect information, by direct policy optimization against worst-case opponents. We prove that when following this optimization, the exploitability of a player's strategy converges asymptotically to zero, and hence when both players employ this optimization, the joint policies converge to a Nash equilibrium. Unlike fictitious play (XFP) and counterfactual regret minimization (CFR), our convergence result pertains to the policies being optimized …
In this paper, we consider the problem of sequential binary hypothesis test in adversary environment based on observations from s sensors, with the caveat that a subset of c sensors is compromised by an adversary, whose observations can be manipulated arbitrarily. We choose the asymptotic Average Sample Number (ASN) required to reach a certain level of error probability as the performance metric of the system. The problem is cast as a game between the detector and the adversary, where the detector aims to optimize the system performance while the adversary tries to deteriorate it. We propose a…
Politecnico di Milano（米兰理工大学）；University of Southampton（南安普顿大学）
We study an information-structure design problem (a.k.a. persuasion) with a single sender and multiple receivers with actions of a priori unknown types, independently drawn from action-specific marginal distributions. As in the standard Bayesian persuasion model, the sender has access to additional information regarding the action types, which she can exploit when committing to a (noisy) signaling scheme through which she sends a private signal to each receiver. The novelty of our model is in considering the case where the receivers interact in a sequential game with imperfect information, wit…
In settings with incomplete information, players can find it difficult to coordinate to find states with good social welfare. For example, in financial settings, if a collection of financial firms have limited information about each other's strategies, some large number of them may choose the same high-risk investment in hopes of high returns. While this might be acceptable in some cases, the economy can be hurt badly if many firms make investments in the same risky market segment and it fails. One reason why many firms might end up choosing the same segment is that they do not have informatio…
Regret minimization is a powerful tool for solving large-scale extensive-form games. State-of-the-art methods rely on minimizing regret locally at each decision point. In this work we derive a new framework for regret minimization on sequential decision problems and extensive-form games with general compact convex sets at each decision point and general convex losses, as opposed to prior work which has been for simplex decision points and linear losses. We call our framework laminar regret decomposition. It generalizes the CFR algorithm to this more general setting. Furthermore, our framework …
We study a basic sequential model for the discovery of winning coalitions in a simple game, well known from its use in defining the Shapley-Shubik power index. We derive in a uniform way a family of measures of collective and individual power in simple games, and show that, as for the Shapley-Shubik index, they extend naturally to measures for TU-games. In particular, the individual measures include all weighted semivalues. We single out the simplest measure in our family for more investigation, as it is new to the literature as far as we know. Although it is very different from the Shapley va…
