We consider two-person zero-sum stochastic mean payoff games with perfect information, or BWR-games, given by a digraph $G = (V, E)$, with local rewards $r: E \to \ZZ$, and three types of positions: black $V_B$, white $V_W$, and random $V_R$ forming a partition of $V$. It is a long-standing open question whether a polynomial time algorithm for BWR-games exists, or not, even when $|V_R|=0$. In fact, a pseudo-polynomial algorithm for BWR-games would already imply their polynomial solvability. In this paper, we show that BWR-games with a constant number of random positions can be solved in pseudo-polynomial time. More precisely, in any BWR-game with $|V_R|=O(1)$, a saddle point in uniformly optimal pure stationary strategies can be found in time polynomial in $|V_W|+|V_B|$, the maximum absolute local reward, and the common denominator of the transition probabilities.
In this paper, we provide an effective characterization of all the subgame-perfect equilibria in infinite duration games played on finite graphs with mean-payoff objectives. To this end, we introduce the notion of requirement, and the notion of negotiation function. We establish that the plays that are supported by SPEs are exactly those that are consistent with the least fixed point of the negotiation function. Finally, we show that the negotiation function is piecewise linear, and can be analyzed using the linear algebraic tool box. As a corollary, we prove the decidability of the SPE constr…
We establish that the subgame perfect equilibrium (SPE) threshold problem for mean-payoff games is NP-complete. While the SPE threshold problem was recently shown to be decidable (in doubly exponential time) and NP-hard, its exact worst case complexity was left open.
We examine perfect information stochastic mean-payoff games - a class of games containing as special sub-classes the usual mean-payoff games and parity games. We show that deterministic memoryless strategies that are optimal for discounted games with state-dependent discount factors close to 1 are optimal for priority mean-payoff games establishing a strong link between these two classes.
We consider some well-known families of two-player, zero-sum, perfect information games that can be viewed as special cases of Shapley's stochastic games. We show that the following tasks are polynomial time equivalent: - Solving simple stochastic games. - Solving stochastic mean-payoff games with rewards and probabilities given in unary. - Solving stochastic mean-payoff games with rewards and probabilities given in binary.
We consider zero-sum stochastic games with finite state and action spaces, perfect information, mean payoff criteria, without any irreducibility assumption on the Markov chains associated to strategies (multichain games). The value of such a game can be characterized by a system of nonlinear equations, involving the mean payoff vector and an auxiliary vector (relative value or bias). We develop here a policy iteration algorithm for zero-sum stochastic games with mean payoff, following an idea of two of the authors (Cochet-Terrasson and Gaubert, C. R. Math. Acad. Sci. Paris, 2006). The algorith…
We examine the problem of the existence of optimal deterministic stationary strategiesintwo-players antagonistic (zero-sum) perfect information stochastic games with finitely many states and actions.We show that the existenceof such strategies follows from the existence of optimal deterministic stationarystrategies for some derived one-player games.Thus we reducethe problem from two-player to one-player games (Markov decisionproblems), where usually it is much easier to tackle.The reduction is very general, it holds not only for all possible payoff mappings but alsoin more a general situations…
Secure equilibrium is a refinement of Nash equilibrium, which provides some security to the players against deviations when a player changes his strategy to another best response strategy. The concept of secure equilibrium is specifically developed for assume-guarantee synthesis and has already been applied in this context. Yet, not much is known about its existence in games with more than two players. In this paper, we establish the existence of secure equilibrium in two classes of multi-player perfect information turn-based games: (1) in games with possibly probabilistic transitions, having …
Quantitative games are two-player zero-sum games played on directed weighted graphs. Total-payoff games (that can be seen as a refinement of the well-studied mean-payoff games) are the variant where the payoff of a play is computed as the sum of the weights. Our aim is to describe the first pseudo-polynomial time algorithm for total-payoff games in the presence of arbitrary weights. It consists of a non-trivial application of the value iteration paradigm. Indeed, it requires to study, as a milestone, a refinement of these games, called min-cost reachability games, where we add a reachability o…
In this work we offer an $O(|V|^2 |E|\, W)$ pseudo-polynomial time deterministic algorithm for solving the Value Problem and Optimal Strategy Synthesis in Mean Payoff Games. This improves by a factor $\log(|V|\, W)$ the best previously known pseudo-polynomial time upper bound due to Brim,~\etal The improvement hinges on a suitable characterization of values, and a description of optimal positional strategies, in terms of reweighted Energy Games and Small Energy-Progress Measures.
In a mean-payoff parity game, one of the two players aims both to achieve a qualitative parity objective and to minimize a quantitative long-term average of payoffs (aka. mean payoff). The game is zero-sum and hence the aim of the other player is to either foil the parity objective or to maximize the mean payoff. Our main technical result is a pseudo-quasi-polynomial algorithm for solving mean-payoff parity games. All algorithms for the problem that have been developed for over a decade have a pseudo-polynomial and an exponential factors in their running times; in the running time of our algor…
A zero-sum two-person Perfect Information Semi-Markov game (PISMG) under limiting ratio average payoff has a value and both the maximiser and the minimiser have optimal pure semi-stationary strategies. We arrive at the result by first fixing an arbitrary initial state and forming the matrix of undiscounted payoffs corresponding to each pair of pure stationary strategies of the two players and proving that this matrix has a pure saddle point.
We consider zero-sum stochastic games with perfect information and finitely many states and actions. The payoff is computed by a function which associates to each infinite sequence of states and actions a real number. We prove that if the payoff function is both shift-invariant and submixing, then the game is half-positional, i.e. the first player has an optimal strategy which is both deterministic and stationary. This result relies on the existence of epsilon-subgame-perfect strategies in shift-invariant games, a second contribution of the paper. The techniques can be used to establish a thir…
Mean-payoff games (MPGs) are infinite duration two-player zero-sum games played on weighted graphs. Under the hypothesis of perfect information, they admit memoryless optimal strategies for both players and can be solved in NP-intersect-coNP. MPGs are suitable quantitative models for open reactive systems. However, in this context the assumption of perfect information is not always realistic. For the partial-observation case, the problem that asks if the first player has an observation-based winning strategy that enforces a given threshold on the mean-payoff, is undecidable. In this paper, we …
In this paper, we propose a pseudo polynomial size LP formulation for finding a payoff vector in the least core of a weighted voting game. The numbers of variables and constraints in our formulation are both bounded by $\mbox{O}(n W_+)$, where $n$ is the number of players and $W_+$ is the total sum of (integer) voting weights. When we employ our formulation, a commercial LP solver calculates a payoff vector in the least core of practical weighted voting games in a few seconds. We also extend our approach to vector weighted voting games.
This paper presents a technique for approximating, up to any precision, the set of subgame-perfect equilibria (SPE) in discounted repeated games. The process starts with a single hypercube approximation of the set of SPE. Then the initial hypercube is gradually partitioned on to a set of smaller adjacent hypercubes, while those hypercubes that cannot contain any point belonging to the set of SPE are simultaneously withdrawn. Whether a given hypercube can contain an equilibrium point is verified by an appropriate mathematical program. Three different formulations of the algorithm for both appro…
The assumptions of necessary rationality and necessary knowledge of strategies, also known as perfect prediction, lead to at most one surviving outcome, immune to the knowledge that the players have of them. Solutions concepts implementing this approach have been defined on both dynamic games with perfect information and no ties, the Perfect Prediction Equilibrium, and strategic games with no ties, the Perfectly Transparent Equilibrium. In this paper, we generalize the Perfectly Transparent Equilibrium to games in extensive form with imperfect information and no ties. Both the Perfect Predicti…
In 1953, Kuhn showed that every sequential game has a Nash equilibrium by showing that a procedure, named ``backward induction'' in game theory, yields a Nash equilibrium. It actually yields Nash equilibria that define a proper subclass of Nash equilibria. In 1965, Selten named this proper subclass subgame perfect equilibria. In game theory, payoffs are rewards usually granted at the end of a game. Although traditional game theory mainly focuses on real-valued payoffs that are implicitly ordered by the usual total order over the reals, works of Simon or Blackwell already involved partially ord…
We consider 2-player stochastic games with perfectly observed actions, and study the limit, as the discount factor goes to one, of the equilibrium payoffs set. In the usual setup where current states are observed by the players, we show that the set of stationary equilibrium payoffs always converges, and provide a simple example where the set of equilibrium payoffs has no limit. We then introduce the more general model of hidden stochastic game, where the players publicly receive imperfect signals over current states. In this setup we present an example where not only the limit set of equilibr…
Through a stochastic control theoretic approach, we analyze reputation games where a strategic long-lived player acts in a sequential repeated game against a collection of short-lived players. The key assumption in our model is that the information of the short-lived players is nested in that of the long-lived player. This nested information structure is obtained through an appropriate monitoring structure. Under this monitoring structure, we show that, given mild assumptions, the set of Perfect Bayesian Equilibrium payoffs coincide with Markov Perfect Equilibrium payoffs, and hence a dynamic …
We present a novel coalgebraic formulation of infinite extensive games. We define both the game trees and the strategy profiles by possibly infinite systems of corecursive equations. Certain strategy profiles are proved to be subgame perfect equilibria using a novel proof principle of predicate coinduction. We characterize all subgame perfect equilibria for the dollar auction game. The economically interesting feature is that in order to prove these results we do not need to rely on continuity assumptions on the payoffs which amount to discounting the future. In particular, we prove a form of …
Subgame perfect equilibria are specific Nash equilibria in perfect information games in extensive form. They are important because they relate to the rationality of the players. They always exist in infinite games with continuous real-valued payoffs, but may fail to exist even in simple games with slightly discontinuous payoffs. This article considers only games whose outcome functions are measurable in the Hausdorff difference hierarchy of the open sets (\textit{i.e.} $Δ^0_2$ when in the Baire space), and it characterizes the families of linear preferences such that every game using these pre…
We study the algorithm of Gurvich, Khachyian and Karzanov (GKK algorithm) when it is ran over mean-payoff games with no simple cycle of weight zero. We propose a new symmetric analysis, lowering the $O(n^2 N)$ upper-bound of Pisaruk on the number of iterations down to $N + Ep + Em$, which is smaller than $nN$, where $n$ is the number of vertices, $N$ is the largest absolute value of a weight, and $Ep$ and $Em$ are respectively the largest finite energy and dual-energy values of the game. Since each iteration is computed in $O(m)$, this improves on the state of the art pseudopolynomial $O(mnN)$…
The paper proposes a natural measure space of zero-sum perfect information games with upper semicontinuous payoffs. Each game is specified by the game tree, and by the assignment of the active player and of the capacity to each node of the tree. The payoff in a game is defined as the infimum of the capacity over the nodes that have been visited during the play. The active player, the number of children, and the capacity are drawn from a given joint distribution independently across the nodes. We characterize the cumulative distribution function of the value $v$ using the fixed points of the so…
It is well-known that for infinitely repeated games, there are computable strategies that have best responses, but no computable best responses. These results were originally proved for either specific games (e.g., Prisoner's dilemma), or for classes of games satisfying certain conditions not known to be both necessary and sufficient. We derive a complete characterization in the form of simple necessary and sufficient conditions for the existence of a computable strategy without a computable best response under limit-of-means payoff. We further refine the characterization by requiring the stra…
We study the problem of finding equilibrium strategies in multi-agent games with incomplete payoff information, where the payoff matrices are only known to the players up to some bounded uncertainty sets. In such games, an ex-post equilibrium characterizes equilibrium strategies that are robust to the payoff uncertainty. When the game is one-shot, we show that in zero-sum polymatrix games, an ex-post equilibrium can be computed efficiently using linear programming. We further extend the notion of ex-post equilibrium to stochastic games, where the game is played repeatedly in a sequence of stag…
We consider the problem in which n items arrive to a market sequentially over time, where two agents compete to choose the best possible item. When an agent selects an item, he leaves the market and obtains a payoff given by the value of the item, which is represented by a random variable following a known distribution with support contained in [0, 1]. We consider two different settings for this problem. In the first one, namely competitive selection problem with no recall, agents observe the value of each item upon its arrival and decide whether to accept or reject it, in which case they will…
Pseudo-games are a natural and well-known generalization of normal-form games, in which the actions taken by each player affect not only the other players' payoffs, as in games, but also the other players' strategy sets. The solution concept par excellence for pseudo-games is the generalized Nash equilibrium (GNE), i.e., a strategy profile at which each player's strategy is feasible and no player can improve their payoffs by unilaterally deviating to another strategy in the strategy set determined by the other players' strategies. The computation of GNE in pseudo-games has long been a problem …
A valuation for a player in a game in extensive form is an assignment of numeric values to the players moves. The valuation reflects the desirability moves. We assume a myopic player, who chooses a move with the highest valuation. Valuations can also be revised, and hopefully improved, after each play of the game. Here, a very simple valuation revision is considered, in which the moves made in a play are assigned the payoff obtained in the play. We show that by adopting such a learning process a player who has a winning strategy in a win-lose game can almost surely guarantee a win in a repeate…
The paper proposes a novel neuroevolutionary method (NESG) for calculating leader's payoff in Stackelberg Security Games. The heart of NESG is strategy evaluation neural network (SENN). SENN is able to effectively evaluate leader's strategies against an opponent who may potentially not behave in a perfectly rational way due to certain cognitive biases or limitations. SENN is trained on historical data and does not require any direct prior knowledge regarding the follower's target preferences, payoff distribution or bounded rationality model. NESG was tested on a set of 90 benchmark games inspi…
Semidefinite programming can be considered over any real closed field, including fields of Puiseux series equipped with their nonarchimedean valuation. Nonarchimedean semidefinite programs encode parametric families of classical semidefinite programs, for sufficiently large values of the parameter. Recently, a correspondence has been established between nonarchimedean semidefinite programs and stochastic mean payoff games with perfect information. This correspondence relies on tropical geometry. It allows one to solve generic nonarchimedean semidefinite feasibility problems, of large scale, by…
In the framework of finite games in extensive form with perfect information and strict preferences, this paper introduces a new equilibrium concept: the Perfect Prediction Equilibrium (PPE). In the Nash paradigm, rational players consider that the opponent's strategy is fixed while maximizing their payoff. The PPE, on the other hand, models the behavior of agents with an alternate form of rationality that involves a Stackelberg competition with the past. Agents with this form of rationality integrate in their reasoning that they have such accurate logical and predictive skills, that the world …
This paper investigates mixed strategies in dynamic games with perfect information. We present an example to show that a player may obtain higher payoff by playing mixed strategy. By contrast, the main result of the paper shows that every two-player dynamic zero-sum game with nature has the no-mixing property, which implies that mixed strategy is useless in this most classical class of games. As for applications, we show the existence of pure-strategy subgame-perfect equilibria in two-player zero-sum games with nature. Based on the main result, we also prove the existence of a universal subgam…
We consider two-player random extensive form games where the payoffs at the leaves are independently drawn uniformly at random from a given feasible set C. We study the asymptotic distribution of the subgame perfect equilibrium outcome for binary-trees with increasing depth in various random (or deterministic) assignments of players to nodes. We characterize the assignments under which the asymptotic distribution concentrates around a point. Our analysis provides a natural way to derive from the asymptotic distribution a novel solution concept for two-player bargaining problems with a solid st…
Two-player mean-payoff Stackelberg games are nonzero-sum infinite duration games played on a bi-weighted graph by Leader (Player 0) and Follower (Player 1). Such games are played sequentially: first, Leader announces her strategy, second, Follower chooses his best-response. If we cannot impose which best-response is chosen by Follower, we say that Follower, though strategic, is adversarial towards Leader. The maximal value that Leader can get in this nonzero-sum game is called the adversarial Stackelberg value (ASV) of the game. We study the robustness of strategies for Leader in these games a…
We introduce a model of sender-receiver stopping games, where the state of the world follows an iid--process throughout the game. At each period, the sender observes the current state, and sends a message to the receiver, suggesting either to stop or to continue. The receiver, only seeing the message but not the state, decides either to stop the game, or to continue which takes the game to the next period. The payoff to each player is a function of the state when the receiver quits, with higher states leading to better payoffs. The horizon of the game can be finite or infinite. We prove existe…
We generalise the hyperplane separation technique (Chatterjee and Velner, 2013) from multi-dimensional mean-payoff to energy games, and achieve an algorithm for solving the latter whose running time is exponential only in the dimension, but not in the number of vertices of the game graph. This answers an open question whether energy games with arbitrary initial credit can be solved in pseudo-polynomial time for fixed dimensions 3 or larger (Chaloupka, 2013). It also improves the complexity of solving multi-dimensional energy games with given initial credit from non-elementary (Brázdil, Jančar,…
We show that the problem of deciding whether in a multi-player perfect information recursive game (i.e. a stochastic game with terminal rewards) there exists a stationary Nash equilibrium ensuring each player a certain payoff is Existential Theory of the Reals complete. Our result holds for acyclic games, where a Nash equilibrium may be computed efficiently by backward induction, and even for deterministic acyclic games with non-negative terminal rewards. We further extend our results to the existence of Nash equilibria where a single player is surely winning. Combining our result with known g…
Zero-determinant (ZD) strategies, a recently found novel class of strategies in repeated games, has attracted much attention in evolutionary game theory. A ZD strategy unilaterally enforces a linear relation between average payoffs of players. Although existence and evolutional stability of ZD strategies have been studied in simple games, their mathematical properties have not been well-known yet. For example, what happens when more than one players employ ZD strategies have not been clarified. In this paper, we provide a general framework for investigating situations where more than one playe…
In a secondary spectrum market primaries set prices for their unused channels to the secondaries. The payoff of a primary depends on the availability of unused channels of its competitors. We consider a model were a primary can acquire its competitor's channel state information (C-CSI) at a cost. We formulate a game between two primaries where each primary decides whether to acquire C-CSI or not and then selects its price based on that. We first characterize the Nash Equilibrium (NE) of this game for a symmetric model where the C-CSI is perfect. We show that the payoff of a primary is independ…
This note studies structural aspects concerning Optimal Positional Strategies (OPSs) in Mean Payoff Games (MPGs), it is a contribution to understanding the relationship between OPSs in MPGs and Small Energy-Progress Measures (SEPMs) in reweighted Energy Games (EGs). Firstly, it is observed that the space of all OPSs, $\texttt{opt}_ΓΣ^M_0$, admits a unique complete decomposition in terms of so-called extremal-SEPM{s} in reweighted EG{s}; this points out what we called the "Energy-Lattice $\mathcal{X}^*_Γ$ of $\texttt{opt}_ΓΣ^M_0$". Secondly, it is offered a pseudo-polynomial total-time recursiv…
Motivated by applications to data networks where fast convergence is essential, we analyze the problem of learning in generic N-person games that admit a Nash equilibrium in pure strategies. Specifically, we consider a scenario where players interact repeatedly and try to learn from past experience by small adjustments based on local - and possibly imperfect - payoff information. For concreteness, we focus on the so-called "hedge" variant of the exponential weights algorithm where players select an action with probability proportional to the exponential of the action's cumulative payoff over t…
We consider Gillette's two-person zero-sum stochastic games with perfect information. For each $k \in \ZZ_+$ we introduce an effective reward function, called $k$-total. For $k = 0$ and $1$ this function is known as {\it mean payoff} and {\it total reward}, respectively. We restrict our attention to the deterministic case. For all $k$, we prove the existence of a saddle point which can be realized by uniformly optimal pure stationary strategies. We also demonstrate that $k$-total reward games can be embedded into $(k+1)$-total reward games.
This paper considers a class of experimentation games with Lévy bandits encompassing those of Bolton and Harris (1999) and Keller, Rady and Cripps (2005). Its main result is that efficient (perfect Bayesian) equilibria exist whenever players' payoffs have a diffusion component. Hence, the trade-offs emphasized in the literature do not rely on the intrinsic nature of bandit models but on the commonly adopted solution concept (MPE). This is not an artifact of continuous time: we prove that efficient equilibria arise as limits of equilibria in the discrete-time game. Furthermore, it suffices to r…
In constructing an econometric or statistical model, we pick relevant features or variables from many candidates. A coalitional game is set up to study the selection problem where the players are the candidates and the payoff function is a performance measurement in all possible modeling scenarios. Thus, in theory, an irrelevant feature is equivalent to a dummy player in the game, which contributes nothing to all modeling situations. The hypothesis test of zero mean contribution is the rule to decide a feature is irrelevant or not. In our mechanism design, the end goal perfectly matches the ex…
