Partial-monitoring games constitute a mathematical framework for sequential decision making problems with imperfect feedback: The learner repeatedly chooses an action, opponent responds with an outcome, and then the learner suffers a loss and receives a feedback signal, both of which are fixed functions of the action and the outcome. The goal of the learner is to minimize his total cumulative loss. We make progress towards the classification of these games based on their minimax expected regret. Namely, we classify almost all games with two outcomes and finite number of actions: We show that their minimax expected regret is either zero, $\widetildeΘ(\sqrt{T})$, $Θ(T^{2/3})$, or $Θ(T)$ and we give a simple and efficiently computable classification of these four classes of games. Our hope is that the result can serve as a stepping stone toward classifying all finite partial-monitoring games.
If in a signaling game the receiver expects to gain no information by monitoring the signal of the sender, then when a cost to monitor is implemented he will never pay that cost regardless of his off-path beliefs. This is the argument of a recent paper by T. Denti (2021). However, which pooling equilibrium does a receiver anticipate to gain no information through monitoring? This paper seeks to prove that given a sufficiently small cost to monitor any pooling equilibrium with a non-zero index will survive close to the original equilibrium.
We consider an attacker-operator game for monitoring a large-scale network that is comprised on components that differ in their criticality levels. In this zero-sum game, the operator seeks to position a limited number of sensors to monitor the network against an attacker who strategically targets a network component. The operator (resp. attacker) seeks to minimize (resp. maximize) the network loss. To study the properties of mixed-strategy Nash Equilibria of this game, we first study two simple instances: (i) When component sets monitored by individual sensor locations are mutually disjoint; …
Studying continuous time counterpart of some discrete time dynamics is now a standard and fruitful technique, as some properties hold in both setups. In game theory, this is usually done by considering differential games on Euclidean spaces. This allows to infer properties on the convergence of values of a repeated game, to deal with the various concepts of approachability, etc. In this paper, we introduce a specific but quite abstract differential game defined on the Wasserstein space of probability distributions and we prove the existence of its value. Going back to the discrete time dynamic…
Arizona State University（亚利桑那州立大学）；WS AI Labs（WS人工智能实验室）
We consider scenarios where a worker robot, who may be unaware of the human's exact expectations, may have the incentive to deviate from a preferred plan (e.g. safe but costly) when a human supervisor is not monitoring it. On the other hand, continuous monitoring of the robot's behavior is often difficult for humans because it costs them valuable resources (e.g., time, cognitive overload, etc.). Thus, to optimize the cost of monitoring while ensuring the robots follow the {\em safe} behavior and to assist the human to deal with the possible unsafe robots, we model this problem in a game-theore…
Parity games have been broadly studied in recent years for their applications to controller synthesis and verification. In practice, partial solvers for parity games that execute in polynomial time, while incomplete, can solve most games in publicly available benchmark suites. In this paper, we combine those partial solvers with the classical recursive algorithm for parity games due to Zielonka. We also extend partial solvers to generalized parity games that are games with conjunction of parity objectives. We have implemented those algorithms and evaluated them on a large set of benchmarks pro…
Disentangling trust from cooperation: Evolution of trust as reduced monitoring in social dilemmas
University of Lausanne（洛桑大学）；Teesside University（蒂斯赛德大学）；Université libre de Bruxelles（布鲁塞尔自由大学）；Vrije Universiteit Brussel（布鲁塞尔自由大学）；University of Stirling（斯特灵大学）
Value methods for solving stochastic games with partial observability model the uncertainty about states of the game as a probability distribution over possible states. The dimension of this belief space is the number of states. For many practical problems, for example in security, there are exponentially many possible states which causes an insufficient scalability of algorithms for real-world problems. To this end, we propose an abstraction technique that addresses this issue of the curse of dimensionality by projecting high-dimensional beliefs to characteristic vectors of significantly lowe…
在完全监控的可逼近性中，已知有两种类型的条件对于凸集是等价的：原始条件和对偶条件。原始的形式是：一个集合C是可接近的当且仅当所有包含的半空间在一次博弈中是可接近的;而对偶的形式是：一个凸集C是可接近的当且仅当它与所有具有某种形式的支付集相交。我们考虑部分监控博弈中的可接近性。在以前的工作（Perchet 2011; Mannor et al. 2011）中，我们提供了可逼近凸集的对偶特征;我们还展示了C是多胞形的情况下的有效策略。本文给出了凸集可部分监控逼近的基本条件。它们依赖于修改后的报酬函数，并导致可接近性策略，基于修改后的支付函数，进行类似于Blackwell（1956）的策略的预测;这是在对比与以前研究的策略，在这种情况下，主要依赖于信号结构，旨在估计以及所收到的信号的分布。我们的结果推广了Kohlberg 1975的经典结果（也可参见Mertens et al. 1994），并适用于具有任意信号结构的博弈以及任意凸集。
We consider an example of stochastic games with partial, asymmetric and non-classical information. We obtain relevant equilibrium policies using a new approach which allows managing the belief updates in a structured manner. Agents have access only to partial information updates, and our approach is to consider optimal open loop control until the information update. The agents continuously control the rates of their Poisson search clocks to acquire the locks, the agent to get all the locks before others would get reward one. However, the agents have no information about the acquisition status …
This paper studies repeated games where two players play multiple duopolistic games simultaneously (multimarket contact). A key assumption is that each player receives a noisy and private signal about the other's actions (private monitoring or observation errors). There has been no game-theoretic support that multimarket contact facilitates collusion or not, in the sense that more collusive equilibria in terms of per-market profits exist than those under a benchmark case of one market. An equilibrium candidate under the benchmark case is belief-free strategies. We are the first to construct a …
In this article we analyze a partial-information Nash Q-learning algorithm for a general 2-player stochastic game. Partial information refers to the setting where a player does not know the strategy or the actions taken by the opposing player. We prove convergence of this partially informed algorithm for general 2-player games with finitely many states and actions, and we confirm that the limiting strategy is in fact a full-information Nash equilibrium. In implementation, partial information offers simplicity because it avoids computation of Nash equilibria at every time step. In contrast, ful…
Naval Center for Applied Research in A.I.（海军应用人工智能中心）；U.S. Naval Research Laboratory（美国海军研究实验室）；Oberlin College（欧柏林学院）
Reinforcement learning (RL) is successful at learning to play games where the entire environment is visible. However, RL approaches are challenged in complex games like Starcraft II and in real-world environments where the entire environment is not visible. In these more complex games with more limited visual information, agents must choose where to look and how to optimally use their limited visual information in order to succeed at the game. We verify that with a relatively simple model the agent can learn where to look in scenarios with a limited visual bandwidth. We develop a method for ma…
We study the following repeated non-atomic routing game. In every round, nature chooses a state in an i.i.d. manner according to a publicly known distribution, which influences link latency functions. The system planner makes private route recommendations to participating agents, which constitute a fixed fraction, according to a publicly known signaling strategy. The participating agents choose between obeying or not obeying the recommendation according to cumulative regret of the participating agent population in the previous round. The non-participating agents choose route according to myopi…
The decision making and management of many engineering networks involves multiple parties with conflicting interests, while each party is constituted with multiple agents. Such problems can be casted as a multi-cluster game. Each cluster is treated as a self-interested player in a non-cooperative game where agents in the same cluster cooperate together to optimize the payoff function of the cluster. In a large-scale network, the information of agents in a cluster can not be available immediately for agents beyond this cluster, which raise challenges to the existing Nash equilibrium seeking alg…
