Graph games lie at the algorithmic core of many automated design problems in computer science. These are games usually played between two players on a given graph, where the players keep moving a token along the edges according to pre-determined rules, and the winner is decided based on the infinite path traversed by the token from a given initial position. In bidding games, the players initially get some monetary budgets which they need to use to bid for the privilege of moving the token at each step. Each round of bidding affects the players' available budgets, which is the only form of update that the budgets experience. We introduce bidding games with charging where the players can additionally improve their budgets during the game by collecting vertex-dependent charges. Unlike traditional bidding games (where all charges are zero), bidding games with charging allow non-trivial recurrent behaviors. We show that the central property of traditional bidding games generalizes to bidding games with charging: For each vertex there exists a threshold ratio, which is the necessary and sufficient fraction of the total budget for winning the game from that vertex. While the thresholds of traditional bidding games correspond to unique fixed points of linear systems of equations, in games with charging, these fixed points are no longer unique. This significantly complicates the proof of existence and the algorithmic computation of thresholds for infinite-duration objectives. We also provide the lower complexity bounds for computing thresholds for Rabin and Streett objectives, which are the first known lower bounds in any form of bidding games (with or without charging), and we solve the following repair problem for safety and reachability games that have unsatisfiable objectives: Can we distribute a given amount of charge to the players in a way such that the objective can be satisfied?
This paper considers an electric vehicle charging scheduling setting where vehicle users can reserve charging time in advance at a charging station. In this setting, users are allowed to explicitly express their preferences over different start times and the length of charging periods for charging their vehicles. The goal is to compute optimal charging schedules which maximize the social welfare of all users given their time preferences and the state of charge of their vehicles. Assuming that users are self-interested agents who may behave strategically to advance their own benefits rather tha…
Richman games are zero-sum games, where in each turn players bid in order to determine who will play next [Lazarus et al.'99]. We extend the theory to impartial general-sum two player games called \emph{bidding games}, showing the existence of pure subgame-perfect equilibria (PSPE). In particular, we show that PSPEs form a semilattice, with a unique and natural \emph{Bottom Equilibrium}. Our main result shows that if only two actions available to the players in each node, then the Bottom Equilibrium has additional properties: (a) utilities are monotone in budget; (b) every outcome is Pareto-ef…
In two-player games on graphs, the players move a token through a graph to produce an infinite path, which determines the winner or payoff of the game. We study {\em bidding games} in which the players bid for the right to move the token. Several bidding rules were studied previously. In {\em Richman} bidding, in each round, the players simultaneously submit bids, and the higher bidder moves the token and pays the other player. {\em Poorman} bidding is similar except that the winner of the bidding pays the "bank" rather than the other player. {\em Taxman} bidding spans the spectrum between Ric…
Two-player zero-sum "graph games" are a central model, which proceeds as follows. A token is placed on a vertex of a graph, and the two players move it to produce an infinite "play", which determines the winner or payoff of the game. Traditionally, the players alternate turns in moving the token. In "bidding games", however, the players have budgets and in each turn, an auction (bidding) determines which player moves the token. So far, bidding games have only been studied as full-information games. In this work we initiate the study of partial-information bidding games: we study bidding games …
In two-player games on graphs, the players move a token through a graph to produce an infinite path, which determines the winner or payoff of the game. Such games are central in formal verification since they model the interaction between a non-terminating system and its environment. We study {\em bidding games} in which the players bid for the right to move the token. Two bidding rules have been defined. In {\em Richman} bidding, in each round, the players simultaneously submit bids, and the higher bidder moves the token and pays the other player. {\em Poorman} bidding is similar except that …
In a two-player zero-sum graph game the players move a token throughout a graph to produce an infinite path, which determines the winner or payoff of the game. Traditionally, the players alternate turns in moving the token. In {\em bidding games}, however, the players have budgets, and in each turn, we hold an "auction" (bidding) to determine which player moves the token: both players simultaneously submit bids and the higher bidder moves the token. The bidding mechanisms differ in their payment schemes. Bidding games were largely studied with variants of {\em first-price} bidding in which onl…
Two-player games on graphs are widely studied in formal methods as they model the interaction between a system and its environment. The game is played by moving a token throughout a graph to produce an infinite path. There are several common modes to determine how the players move the token through the graph; e.g., in turn-based games the players alternate turns in moving the token. We study the {\em bidding} mode of moving the token, which, to the best of our knowledge, has never been studied in infinite-duration games. The following bidding rule was previously defined and called Richman bidd…
We study zero-sum (combinatorial) games, within the framework of so-called Richman auctions (Lazarus et al. 1996) namely, we modify the alternating play scoring ruleset Cumulative Subtraction (CS) (Cohensius et al. 2019), to a discrete bidding scheme (similar to Develin and Payne 2010). Players bid to move and the player with the highest bid wins the move, and hands over the winning bidding amount to the other player. The new game is dubbed Bidding Cumulative Subtraction (BCS). In so-called unitary games, players remove exactly one item out of a single heap of identical items, until the heap i…
Bidding Games on Markov Decision Processes with Quantitative Reachability Objectives
University of Haifa（海法大学）；Masaryk University（马萨里克大学）；Institute of Science and Technology Austria (ISTA)（奥地利科学技术研究所）；IMDEA Software Institute（IMDEA软件研究所）
Reinforcement Learning Algorithms (RLA) are useful machine learning tools to understand how decision makers react to signals. It is known that RLA converge towards the pure Nash Equilibria (NE) of finite congestion games and more generally, finite potential games. For finite congestion games, only separable cost functions are considered. However, non-separable costs, which depend on the choices of all players instead of only those choosing the same resource, may be relevant in some circumstances, like in smart charging games. In this paper, finite congestion games with non-separable costs are …
The connection between games and no-regret algorithms has been widely studied in the literature. A fundamental result is that when all players play no-regret strategies, this produces a sequence of actions whose time-average is a coarse-correlated equilibrium of the game. However, much less is known about equilibrium selection in the case that multiple equilibria exist. In this work, we study the convergence of no-regret bidding algorithms in auctions. Besides being of theoretical interest, bidding dynamics in auctions is an important question from a practical viewpoint as well. We study repea…
IST Austria（奥地利科学技术研究所）；Liverpool University（利物浦大学）
In this paper we introduce and study {\em all-pay bidding games}, a class of two player, zero-sum games on graphs. The game proceeds as follows. We place a token on some vertex in the graph and assign budgets to the two players. Each turn, each player submits a sealed legal bid (non-negative and below their remaining budget), which is deducted from their budget and the highest bidder moves the token onto an adjacent vertex. The game ends once a sink is reached, and \PO pays \PT the outcome that is associated with the sink. The players attempt to maximize their expected outcome. Our games model…
Small operators who take part in secondary wireless spectrum markets typically have strict budget limits. In this paper, we study the bidding problem of a budget constrained operator in repeated secondary spectrum auctions. In existing truthful auctions, truthful bidding is the optimal strategy of a bidder. However, budget limits impact bidding behaviors and make bidding decisions complicated, since bidders may behave differently to avoid running out of money. We formulate the problem as a dynamic auction game between operators, where knowledge of other operators is limited due to the distribu…
Bid optimization for online advertising from single advertiser's perspective has been thoroughly investigated in both academic research and industrial practice. However, existing work typically assume competitors do not change their bids, i.e., the wining price is fixed, leading to poor performance of the derived solution. Although a few studies use multi-agent reinforcement learning to set up a cooperative game, they still suffer the following drawbacks: (1) They fail to avoid collusion solutions where all the advertisers involved in an auction collude to bid an extremely low price on purpose…
The Bidding Games: Reinforcement Learning for MEV Extraction on Polygon Blockchain
Independent Researcher（独立研究者）；Moscow Institute of Physics and Technology（莫斯科物理技术学院）；Skolkovo Institute of Science and Technology（斯克尔科沃科学与技术研究所）；HSE University（俄罗斯伏尔加格勒国立高等经济大学）
We consider the charge scheduling coordination of a fleet of plug-in electric vehicles, developing a hybrid decision-making framework for efficient and profitable usage of the distribution grid. Each charging dynamics, affected by the aggregate behavior of the whole fleet, is modelled as an inter-dependent, mixed-logical-dynamical system. The coordination problem is formalized as a generalized mixed-integer aggregative potential game, and solved via semi-decentralized implementation of a sequential best-response algorithm that leads to an approximated equilibrium of the game.
Financial transmission right (FTR) is an important tool and an especially feature for stopping congestion charges in restructured electricity markets. Participants in the transmission market as players are assumed to be a generation company (Gencos) which also take part in an energy market and able to buy their require FTRs. In this regard, there are two types of FTR: obligation or option. There are three main questions which immediately arise for each player who is placed in the market. First, which type of FTR is the best choice second, how much power is needed to generate by each player and…
Incorporating Bounded Rationality into Electric Vehicle Highway Charging Decisions: A Bayesian Game Analysis
The Chinese University of Hong Kong, Shenzhen（香港中文大学（深圳））；Shenzhen Institute of Artificial Intelligence and Robotics for Society（深圳市人工智能与机器人研究院）；Shenzhen Key Laboratory of Crowd Intelligence Empowered Low-Carbon Energy Network（深圳市 crowd intelligence 赋能低碳能源网络重点实验室）
Spot electricity markets are considered under a Game-Theoretic framework, where risk averse players submit orders to the market clearing mechanism to maximise their own utility. Consistent with the current practice in Europe, the market clearing mechanism is modelled as a Social Welfare Maximisation problem, with zonal pricing, and we consider inflexible demand, physical constraints of the electricity grid, and capacity-constrained producers. A novel type of non-parametric risk aversion based on a defined worst case scenario is introduced, and this reduces the dimensionality of the strategy va…
This paper introduces a "green" routing game between multiple logistic operators (players), each owning a mixed fleet of internal combustion engine vehicle (ICEV) and electric vehicle (EV) trucks. Each player faces the cost of delayed delivery (due to charging requirements of EVs) and a pollution cost levied on the ICEVs. This cost structure models: 1) limited battery capacity of EVs and their charging requirement; 2) shared nature of charging facilities; 3) pollution cost levied by regulatory agency on the use of ICEVs. We characterize Nash equilibria of this game and derive a condition for i…
This paper presents the equilibrium analysis of a game composed of heterogeneous electric vehicles (EVs) and a power distribution system operator (DSO) as the players, and charging station operators (CSOs) and a transportation network operator (TNO) as coordinators. Each EV tries to pick a charging station as its destination and a route to get there at the same time. However, the traffic and electrical load congestion on the roads and charging stations lead to the interdependencies between the optimal decisions of EVs. CSOs and the TNO need to apply some tolling to control such congestion. On …
