This note studies structural aspects concerning Optimal Positional Strategies (OPSs) in Mean Payoff Games (MPGs), it is a contribution to understanding the relationship between OPSs in MPGs and Small Energy-Progress Measures (SEPMs) in reweighted Energy Games (EGs). Firstly, it is observed that the space of all OPSs, $\texttt{opt}_ΓΣ^M_0$, admits a unique complete decomposition in terms of so-called extremal-SEPM{s} in reweighted EG{s}; this points out what we called the "Energy-Lattice $\mathcal{X}^*_Γ$ of $\texttt{opt}_ΓΣ^M_0$". Secondly, it is offered a pseudo-polynomial total-time recursive procedure for enumerating (w/o repetitions) all the elements of $\mathcal{X}^*_Γ$, and for computing the corresponding partitioning of $\texttt{opt}_ΓΣ^M_0$. It is observed that the corresponding recursion tree defines an additional lattice $\mathcal{B}^*_Γ$, whose elements are certain subgames $Γ'\subseteq Γ$ that we call basic subgames. The extremal-SEPMs of a given \MPG $Γ$ coincide with the least-SEPMs of the basic subgames of $Γ$; so, $\mathcal{X}^*_Γ$ is the energy-lattice comprising all and only the least-SEPMs of the \emph{basic} subgames of $Γ$. The complexity of the proposed enumeration for both $\mathcal{B}^*_Γ$ and $\mathcal{X}^*_Γ$ is $O(|V|^3|E|W |\mathcal{B}^*_Γ|)$ total time and $O(|V||E|)+Θ\big(|E| \mathcal{B}^*_Γ|\big)$ working space. Finally, it is constructed an \MPG $Γ$ for which $|\mathcal{B}^*_Γ| > |\mathcal{X}^*_Γ|$, this proves that $\mathcal{B}^*_Γ$ and $\mathcal{X}^*_Γ$ are not isomorphic.
We examine perfect information stochastic mean-payoff games - a class of games containing as special sub-classes the usual mean-payoff games and parity games. We show that deterministic memoryless strategies that are optimal for discounted games with state-dependent discount factors close to 1 are optimal for priority mean-payoff games establishing a strong link between these two classes.
We study the problem of finding equilibrium strategies in multi-agent games with incomplete payoff information, where the payoff matrices are only known to the players up to some bounded uncertainty sets. In such games, an ex-post equilibrium characterizes equilibrium strategies that are robust to the payoff uncertainty. When the game is one-shot, we show that in zero-sum polymatrix games, an ex-post equilibrium can be computed efficiently using linear programming. We further extend the notion of ex-post equilibrium to stochastic games, where the game is played repeatedly in a sequence of stag…
In a mean-payoff parity game, one of the two players aims both to achieve a qualitative parity objective and to minimize a quantitative long-term average of payoffs (aka. mean payoff). The game is zero-sum and hence the aim of the other player is to either foil the parity objective or to maximize the mean payoff. Our main technical result is a pseudo-quasi-polynomial algorithm for solving mean-payoff parity games. All algorithms for the problem that have been developed for over a decade have a pseudo-polynomial and an exponential factors in their running times; in the running time of our algor…
We consider some well-known families of two-player, zero-sum, perfect information games that can be viewed as special cases of Shapley's stochastic games. We show that the following tasks are polynomial time equivalent: - Solving simple stochastic games. - Solving stochastic mean-payoff games with rewards and probabilities given in unary. - Solving stochastic mean-payoff games with rewards and probabilities given in binary.
We prove that to find optimal positional strategies for stochastic mean payoff games when the value of every state of the game is known, in general, is as hard as solving such games tout court. This answers a question posed by Daniel Andersson and Peter Bro Miltersen.
In this work we offer an $O(|V|^2 |E|\, W)$ pseudo-polynomial time deterministic algorithm for solving the Value Problem and Optimal Strategy Synthesis in Mean Payoff Games. This improves by a factor $\log(|V|\, W)$ the best previously known pseudo-polynomial time upper bound due to Brim,~\etal The improvement hinges on a suitable characterization of values, and a description of optimal positional strategies, in terms of reweighted Energy Games and Small Energy-Progress Measures.
We introduce an algorithm which solves mean payoff games in polynomial time on average, assuming the distribution of the games satisfies a flip invariance property on the set of actions associated with every state. The algorithm is a tropical analogue of the shadow-vertex simplex algorithm, which solves mean payoff games via linear feasibility problems over the tropical semiring $(\mathbb{R} \cup \{-\infty\}, \max, +)$. The key ingredient in our approach is that the shadow-vertex pivoting rule can be transferred to tropical polyhedra, and that its computation reduces to optimal assignment prob…
We establish that the subgame perfect equilibrium (SPE) threshold problem for mean-payoff games is NP-complete. While the SPE threshold problem was recently shown to be decidable (in doubly exponential time) and NP-hard, its exact worst case complexity was left open.
Quantitative games are two-player zero-sum games played on directed weighted graphs. Total-payoff games (that can be seen as a refinement of the well-studied mean-payoff games) are the variant where the payoff of a play is computed as the sum of the weights. Our aim is to describe the first pseudo-polynomial time algorithm for total-payoff games in the presence of arbitrary weights. It consists of a non-trivial application of the value iteration paradigm. Indeed, it requires to study, as a milestone, a refinement of these games, called min-cost reachability games, where we add a reachability o…
In mean-payoff games, the objective of the protagonist is to ensure that the limit average of an infinite sequence of numeric weights is nonnegative. In energy games, the objective is to ensure that the running sum of weights is always nonnegative. Multi-mean-payoff and multi-energy games replace individual weights by tuples, and the limit average (resp. running sum) of each coordinate must be (resp. remain) nonnegative. These games have applications in the synthesis of resource-bounded processes with multiple resources. We prove the finite-memory determinacy of multi-energy games and show the…
We prove that every repeated game with countably many players, finite action sets, and tail-measurable payoffs admits an $ε$-equilibrium, for every $ε> 0$.
Evolutionarily stable strategy (ESS) is a key concept in evolutionary game theory. ESS provides an evolutionary stability criterion for biological, social and economical behaviors. In this paper, we develop a new approach to evaluate ESS in symmetric two player games with fuzzy payoffs. Particularly, every strategy is assigned a fuzzy membership that describes to what degree it is an ESS in presence of uncertainty. The fuzzy set of ESS characterize the nature of ESS. The proposed approach avoids loss of any information that happens by the defuzzification method in games and handles uncertainty…
In this paper, we study one-player and two-player energy mean-payoff games. Energy mean-payoff games are games of infinite duration played on a finite graph with edges labeled by 2-dimensional weight vectors. The objective of the first player (the protagonist) is to satisfy an energy objective on the first dimension and a mean-payoff objective on the second dimension. We show that optimal strategies for the first player may require infinite memory while optimal strategies for the second player (the antagonist) do not require memory. In the one-player case (where only the first player has choic…
It is well known that the rock-paper-scissors game has no pure saddle point. We show that this holds more generally: A symmetric two-player zero-sum game has a pure saddle point if and only if it is not a generalized rock-paper-scissors game. Moreover, we show that every finite symmetric quasiconcave two-player zero-sum game has a pure saddle point. Further sufficient conditions for existence are provided. We apply our theory to a rich collection of examples by noting that the class of symmetric two-player zero-sum games coincides with the class of relative payoff games associated with symmetr…
We study nondeterministic strategies in parity games with the aim of computing a most permissive winning strategy. Following earlier work, we measure permissiveness in terms of the average number/weight of transitions blocked by the strategy. Using a translation into mean-payoff parity games, we prove that the problem of computing (the permissiveness of) a most permissive winning strategy is in NP intersected coNP. Along the way, we provide a new study of mean-payoff parity games. In particular, we prove that the opponent player has a memoryless optimal strategy and give a new algorithm for so…
We study the problem of finding robust equilibria in multiplayer concurrent games with mean payoff objectives. A $(k,t)$-robust equilibrium is a strategy profile such that no coalition of size $k$ can improve the payoff of one its member by deviating, and no coalition of size $t$ can decrease the payoff of other players. We are interested in pure equilibria, that is, solutions that can be implemented using non-randomized strategies. We suggest a general transformation from multiplayer games to two-player games such that pure equilibria in the first game correspond to winning strategies in the …
Traditionally quantitative games such as mean-payoff games and discount sum games have two players -- one trying to maximize the payoff, the other trying to minimize it. The associated decision problem, "Can Eve (the maximizer) achieve, for example, a positive payoff?" can be thought of as one player trying to attain a payoff in the interval $(0,\infty)$. In this paper we consider the more general problem of determining if a player can attain a payoff in a finite union of arbitrary intervals for various payoff functions (liminf, mean-payoff, discount sum, total sum). In particular this include…
The Robinson-Goforth topology of swaps in adjoining payoffs elegantly arranges 2x2 ordinal games in accordance with important properties including symmetry, number of dominant strategies and Nash Equilibria, and alignment of interests. Adding payoff families based on Nash Equilibria illustrates an additional aspect of this order and aids visualization of the topology. Making ties through half-swaps not only creates simpler games within the topology, but, in reverse, breaking ties shows the evolution of preferences, yielding a natural ordering for the topology of 2x2 games with ties. An ordinal…
Using Sperner's lemma for modified partition of a simplex we will constructively prove the existence of a Nash equilibrium in a finite strategic game with sequentially locally non-constant payoff functions. We follow the Bishop style constructive mathematics.
We consider 2-player stochastic games with perfectly observed actions, and study the limit, as the discount factor goes to one, of the equilibrium payoffs set. In the usual setup where current states are observed by the players, we show that the set of stationary equilibrium payoffs always converges, and provide a simple example where the set of equilibrium payoffs has no limit. We then introduce the more general model of hidden stochastic game, where the players publicly receive imperfect signals over current states. In this setup we present an example where not only the limit set of equilibr…
In this paper, we study network games where players are involved in information aggregation processes subject to the differential privacy requirement for players' payoff functions. We propose a Laplace linear-quadratic functional perturbation mechanism, which perturbs players' payoff functions with linear-quadratic functions whose coefficients are produced from truncated Laplace distributions. For monotone games, we show that the LLQFP mechanism maintains the concavity property of the perturbed payoff functions and produces a perturbed NE whose distance from the original NE is bounded and adju…
Mean-payoff games are important quantitative models for open reactive systems. They have been widely studied as games of full observation. In this paper we investigate the algorithmic properties of several sub-classes of mean-payoff games where the players have asymmetric information about the state of the game. These games are in general undecidable and not determined according to the classical definition. We show that such games are determined under a more general notion of winning strategy. We also consider mean-payoff games where the winner can be determined by the winner of a finite cycle…
Evolutionary game theory classically investigates which behavioral patterns are evolutionarily successful in a single game. More recently, a number of contributions have studied the evolution of preferences instead: which subjective conceptualizations of a game's payoffs give rise to evolutionarily successful behavior in a single game. Here, we want to extend this existing approach even further by asking: which general patterns of subjective conceptualizations of payoff functions are evolutionarily successful across a class of games. In other words, we will look at evolutionary competition of …
We introduce the concept of attainable sets of payoffs in two-player repeated games with vector payoffs. A set of payoff vectors is called {\em attainable} if player 1 can ensure that there is a finite horizon $T$ such that after time $T$ the distance between the set and the cumulative payoff is arbitrarily small, regardless of what strategy player 2 is using. This paper focuses on the case where the attainable set consists of one payoff vector. In this case the vector is called an attainable vector. We study properties of the set of attainable vectors, and characterize when a specific vector …
We study the deterministic and randomized query complexity of finding approximate equilibria in bimatrix games. We show that the deterministic query complexity of finding an $ε$-Nash equilibrium when $ε< \frac{1}{2}$ is $Ω(k^2)$, even in zero-one constant-sum games. In combination with previous results \cite{FGGS13}, this provides a complete characterization of the deterministic query complexity of approximate Nash equilibria. We also study randomized querying algorithms. We give a randomized algorithm for finding a $(\frac{3 - \sqrt{5}}{2} + ε)$-Nash equilibrium using $O(\frac{k \cdot \log k}…
We introduce and study incentive equilibria for multi-player meanpayoff games. Incentive equilibria generalise well-studied solution concepts such as Nash equilibria and leader equilibria (also known as Stackelberg equilibria). Recall that a strategy profile is a Nash equilibrium if no player can improve his payoff by changing his strategy unilaterally. In the setting of incentive and leader equilibria, there is a distinguished player called the leader who can assign strategies to all other players, referred to as her followers. A strategy profile is a leader strategy profile if no player, exc…
Recent extensions to dynamic games of the well-known fictitious play learning procedure in static games were proved to globally converge to stationary Nash equilibria in two important classes of dynamic games (zero-sum and identical-interest discounted stochastic games). However, those decentralized algorithms need the players to know exactly the model (the transition probabilities and their payoffs at every stage). To overcome these strong assumptions, our paper introduces regularizations of the systems in (Leslie 2020; Baudin 2022) to construct a family of new decentralized learning algorith…
Modeling strategic conflict from a game theoretical perspective involves dealing with epistemic uncertainty. Payoff uncertainty models are typically restricted to simple probability models due to computational restrictions. Recent breakthroughs Artificial Intelligence (AI) research applied to Poker have resulted in novel approximation approaches such as counterfactual regret minimization, that can successfully deal with large-scale imperfect games. By drawing from these ideas, this work addresses the problem of arbitrary continuous payoff distributions. We propose a method, Harsanyi-Counterfac…
In mean-payoff games, the objective of the protagonist is to ensure that the limit average of an infinite sequence of numeric weights is nonnegative. In energy games, the objective is to ensure that the running sum of weights is always nonnegative. Generalized mean-payoff and energy games replace individual weights by tuples, and the limit average (resp. running sum) of each coordinate must be (resp. remain) nonnegative. These games have applications in the synthesis of resource-bounded processes with multiple resources. We prove the finite-memory determinacy of generalized energy games and sh…
In this paper, we provide an effective characterization of all the subgame-perfect equilibria in infinite duration games played on finite graphs with mean-payoff objectives. To this end, we introduce the notion of requirement, and the notion of negotiation function. We establish that the plays that are supported by SPEs are exactly those that are consistent with the least fixed point of the negotiation function. Finally, we show that the negotiation function is piecewise linear, and can be analyzed using the linear algebraic tool box. As a corollary, we prove the decidability of the SPE constr…
We consider concurrent mean-payoff games, a very well-studied class of two-player (player 1 vs player 2) zero-sum games on finite-state graphs where every transition is assigned a reward between 0 and 1, and the payoff function is the long-run average of the rewards. The value is the maximal expected payoff that player 1 can guarantee against all strategies of player 2. We consider the computation of the set of states with value 1 under finite-memory strategies for player 1, and our main results for the problem are as follows: (1) we present a polynomial-time algorithm; (2) we show that whenev…
Two-player games on graphs is central in many problems in formal verification and program analysis such as synthesis and verification of open systems. In this work we consider solving recursive game graphs (or pushdown game graphs) that can model the control flow of sequential programs with recursion. While pushdown games have been studied before with qualitative objectives, such as reachability and $ω$-regular objectives, in this work we study for the first time such games with the most well-studied quantitative objective, namely, mean-payoff objectives. In pushdown games two types of strateg…
We consider zero-sum stochastic games with finite state and action spaces, perfect information, mean payoff criteria, without any irreducibility assumption on the Markov chains associated to strategies (multichain games). The value of such a game can be characterized by a system of nonlinear equations, involving the mean payoff vector and an auxiliary vector (relative value or bias). We develop here a policy iteration algorithm for zero-sum stochastic games with mean payoff, following an idea of two of the authors (Cochet-Terrasson and Gaubert, C. R. Math. Acad. Sci. Paris, 2006). The algorith…
We consider two-person zero-sum stochastic mean payoff games with perfect information, or BWR-games, given by a digraph $G = (V, E)$, with local rewards $r: E \to \ZZ$, and three types of positions: black $V_B$, white $V_W$, and random $V_R$ forming a partition of $V$. It is a long-standing open question whether a polynomial time algorithm for BWR-games exists, or not, even when $|V_R|=0$. In fact, a pseudo-polynomial algorithm for BWR-games would already imply their polynomial solvability. In this paper, we show that BWR-games with a constant number of random positions can be solved in pseudo…
We consider transformations of normal form games by binding preplay offers of players for payments of utility to other players conditional on them playing designated in the offers strategies. The game-theoretic effect of such preplay offers is transformation of the payoff matrix of the game by transferring payoffs between players. Here we analyze and completely characterize the possible transformations of the payoff matrix of a normal form game by sets of preplay offers.
A recent body of experimental literature has studied empirical game-theoretical analysis, in which we have partial knowledge of a game, consisting of observations of a subset of the pure-strategy profiles and their associated payoffs to players. The aim is to find an exact or approximate Nash equilibrium of the game, based on these observations. It is usually assumed that the strategy profiles may be chosen in an on-line manner by the algorithm. We study a corresponding computational learning model, and the query complexity of learning equilibria for various classes of games. We give basic res…
In wireless mesh network (WMN), multiple service providers (SPs) can cooperate to share resources (e.g., relay nodes and spectrum), to serve their collective subscribed customers for better service. As a reward, SPs are able to achieve more individual benefits, i.e., increased revenue or decreased cost, through efficient utilization of shared network resources. However, this cooperation can be realized only if fair allocation of aggregated payoff, which is the sum of the payoff of all the cooperative SPs, can be achieved. We first formulate such cooperation as a coalitional game with transfera…
We consider two-player games played on weighted directed graphs with mean-payoff and total-payoff objectives, two classical quantitative objectives. While for single-dimensional games the complexity and memory bounds for both objectives coincide, we show that in contrast to multi-dimensional mean-payoff games that are known to be coNP-complete, multi-dimensional total-payoff games are undecidable. We introduce conservative approximations of these objectives, where the payoff is considered over a local finite window sliding along a play, instead of the whole play. For single dimension, we show …
The practical scalability of many optimization algorithms for large extensive-form games is often limited by the games' huge payoff matrices. To ameliorate the issue, Zhang and Sandholm (2020) recently proposed a sparsification technique that factorizes the payoff matrix $\mathbf{A}$ into a sparser object $\mathbf{A} = \hat{\mathbf{A}} + \mathbf{U}\mathbf{V}^\top$, where the total combined number of nonzeros of $\hat{\mathbf{A}}$, $\mathbf{U}$, and $\mathbf{V}$ is significantly smaller. Such a factorization can be used in place of the original payoff matrix in many optimization algorithm, such…
Consider a very simple class of (finite) games: after an initial move by nature, each player makes one move. Moreover, the players have common interests: at each node, all the players get the same payoff. We show that the problem of determining whether there exists a joint strategy where each player has an expected payoff of at least r is NP-complete as a function of the number of nodes in the extensive-form representation of the game.
We consider both finite-state game graphs and recursive game graphs (or pushdown game graphs), that can model the control flow of sequential programs with recursion, with multi-dimensional mean-payoff objectives. In pushdown games two types of strategies are relevant: global strategies, that depend on the entire global history; and modular strategies, that have only local memory and thus do not depend on the context of invocation. We present solutions to several fundamental algorithmic questions and our main contributions are as follows: (1) We show that finite-state multi-dimensional mean-pay…
We consider infinite-state turn-based stochastic games of two players, Box and Diamond, who aim at maximizing and minimizing the expected total reward accumulated along a run, respectively. Since the total accumulated reward is unbounded, the determinacy of such games cannot be deduced directly from Martin's determinacy result for Blackwell games. Nevertheless, we show that these games are determined both for unrestricted (i.e., history-dependent and randomized) strategies and deterministic strategies, and the equilibrium value is the same. Further, we show that these games are generally not d…
Mean-payoff games on timed automata are played on the infinite weighted graph of configurations of priced timed automata between two players, Player Min and Player Max, by moving a token along the states of the graph to form an infinite run. The goal of Player Min is to minimize the limit average weight of the run, while the goal of the Player Max is the opposite. Brenguier, Cassez, and Raskin recently studied a variation of these games and showed that mean-payoff games are undecidable for timed automata with five or more clocks. We refine this result by proving the undecidability of mean-payo…
We propose a novel algorithm for the solution of mean-payoff games that merges together two seemingly unrelated concepts introduced in the context of parity games, small progress measures and quasi dominions. We show that the integration of the two notions can be highly beneficial and significantly speeds up convergence to the problem solution. Experiments show that the resulting algorithm performs orders of magnitude better than the asymptotically-best solution algorithm currently known, without sacrificing on the worst-case complexity.
Motivated by the markets operating on fast time scales, we present a framework for online coalitional games with time-varying coalitional values and propose real-time payoff distribution mechanisms. Specifically, we design two online distributed algorithms to track the Shapley value and the core, the two most widely studied payoff distribution criteria in coalitional game theory. We show that the payoff distribution trajectory resulting from our proposed algorithms converges to a neighborhood of the time-varying solutions. We adopt an operator-theoretic perspective to show the convergence of o…
While it is known that shared quantum entanglement can offer improved solutions to a number of purely cooperative tasks for groups of remote agents, controversy remains regarding the legitimacy of quantum games in a competitive setting--in particular, whether they offer any advantage beyond what is achievable using classical resources. We construct a competitive game between four players based on the minority game where the maximal Nash-equilibrium payoff when played with the appropriate quantum resource is greater than that obtainable by classical means, assuming a local hidden variable model…
