In this paper we study a type of games regularized by the relative entropy, where the players' strategies are coupled through a random environment variable. Besides the existence and the uniqueness of equilibria of such games, we prove that the marginal laws of the corresponding mean-field Langevin systems can converge towards the games' equilibria in different settings. As applications, the dynamic games can be treated as games on a random environment when one treats the time horizon as the environment. In practice, our results can be applied to analysing the stochastic gradient descent algorithm for deep neural networks in the context of supervised learning as well as for the generative adversarial networks.
IBM Research（IBM研究院）；Thomas J. Watson Research Center（托马斯·J·沃森研究中心）
The standard risk minimization paradigm of machine learning is brittle when operating in environments whose test distributions are different from the training distribution due to spurious correlations. Training on data from many environments and finding invariant predictors reduces the effect of spurious features by concentrating models on features that have a causal relationship with the outcome. In this work, we pose such invariant risk minimization as finding the Nash equilibrium of an ensemble game among several environments. By doing so, we develop a simple training algorithm that uses be…
In this paper, we study the problem of finding mixed Nash equilibrium for mean-field two-player zero-sum games. Solving this problem requires optimizing over two probability distributions. We consider a quasistatic Wasserstein gradient flow dynamics in which one probability distribution follows the Wasserstein gradient flow, while the other one is always at the equilibrium. Theoretical analysis are conducted on this dynamics, showing its convergence to the mixed Nash equilibrium under mild conditions. Inspired by the continuous dynamics of probability distributions, we derive a quasistatic Lan…
Partially Observable Stochastic Games (POSGs) are the most general and common model of games used in Multi-Agent Reinforcement Learning (MARL). We argue that the POSG model is conceptually ill suited to software MARL environments, and offer case studies from the literature where this mismatch has led to severely unexpected behavior. In response to this, we introduce the Agent Environment Cycle Games (AEC Games) model, which is more representative of software implementation. We then prove it's as an equivalent model to POSGs. The AEC games model is also uniquely useful in that it can elegantly …
The theory of learning in games has extensively studied situations where agents respond dynamically to each other by optimizing a fixed utility function. However, in real situations, the strategic environment varies as a result of past agent choices. Unfortunately, the analysis techniques that enabled a rich characterization of the emergent behavior in static environment games fail to cope with dynamic environment games. To address this, we develop a general framework using probabilistic couplings to extend the analysis of static environment games to dynamic ones. Using this approach, we obtai…
Operating vehicles in adversarial environments require non-conventional planning techniques. A two-player, zero-sum non-cooperative game is introduced, which is solved via a linear program. An extension is proposed to construct networks displaying good representations of the environment characteristics, while offering acceptable results for the technique used. Sensitivity of the solution to the LP solver algorithm is identified. The performances of the planner are finally assessed by comparison with those of conventional planners. Results are used to formulate secondary objectives to the probl…
In the optional prisoner's dilemma (OPD), players can choose to cooperate and defect as usual, but can also abstain as a third possible strategy. This strategy models the players' participation in the game and is a relevant aspect in many settings, e.g. social networks or opinion dynamics where abstention is an option during an election. In this paper, we provide a formulation of the OPD where we consider irrational behaviours in the population inspired by prospect theory. Prospect theory has gained increasing popularity in recent times thanks to its ability to capture aspects such as referenc…
Naval Center for Applied Research in A.I.（海军应用人工智能中心）；U.S. Naval Research Laboratory（美国海军研究实验室）；Oberlin College（欧柏林学院）
Reinforcement learning (RL) is successful at learning to play games where the entire environment is visible. However, RL approaches are challenged in complex games like Starcraft II and in real-world environments where the entire environment is not visible. In these more complex games with more limited visual information, agents must choose where to look and how to optimally use their limited visual information in order to succeed at the game. We verify that with a relatively simple model the agent can learn where to look in scenarios with a limited visual bandwidth. We develop a method for ma…
This paper discusses our investigation into the evolution of cooperative players in an online business environment. We explain our design of an incentive based system with its foundation over binary reputation system whose proportion of reward or punishment is a function of transaction value and the players past history of cooperation. We compare the evolution of cooperation in our setting with non-incentive based environment and our findings show that the incentive based method is more suitable for the evolution of trustworthy players.
The use of game theoretic models has been quite successful in describing various cooperative and non-cooperative optimization problems in networks and other domains of computer systems. In this paper, we study an application of game theoretic models in the domain of distributed system, where nodes play a game to balance the total processing loads among themselves. We have used congestion gaming model, a model of game theory where many agents compete for allocating resources, and studied the existence of Nash Equilibrium for such types of games. As the classical congestion game is known to be P…
We study the quality of outcomes in repeated games when the population of players is dynamically changing and participants use learning algorithms to adapt to the changing environment. Game theory classically considers Nash equilibria of one-shot games, while in practice many games are played repeatedly, and in such games players often use algorithmic tools to learn to play in the given environment. Most previous work on learning in repeated games assumes that the population playing the game is static over time. We analyze the efficiency of repeated games in dynamically changing environments, …
This paper presents an approach that brings together game theory with grammatical inference and discrete abstractions in order to synthesize control strategies for hybrid dynamical systems performing tasks in partially unknown but rule-governed adversarial environments. The combined formulation guarantees that a system specification is met if (a) the true model of the environment is in the class of models inferable from a positive presentation, (b) a characteristic sample is observed, and (c) the task specification is satisfiable given the capabilities of the system (agent) and the environment…
We use a reformulation of compositional game theory to reunite game theory with game semantics, by viewing an open game as the System and its choice of contexts as the Environment. Specifically, the system is jointly controlled by $n \geq 0$ noncooperative players, each independently optimising a real-valued payoff. The goal of the system is to play a Nash equilibrium, and the goal of the environment is to prevent it. The key to this is the realisation that lenses (from functional programming) form a dialectica category, which have an existing game-semantic interpretation. In the second half o…
We introduce open games as a compositional foundation of economic game theory. A compositional approach potentially allows methods of game theory and theoretical computer science to be applied to large-scale economic models for which standard economic tools are not practical. An open game represents a game played relative to an arbitrary environment and to this end we introduce the concept of coutility, which is the utility generated by an open game and returned to its environment. Open games are the morphisms of a symmetric monoidal category and can therefore be composed by categorical compos…
Identifying the actual adversarial threat against a system vulnerability has been a long-standing challenge for cybersecurity research. To determine an optimal strategy for the defender, game-theoretic based decision models have been widely used to simulate the real-world attacker-defender scenarios while taking the defender's constraints into consideration. In this work, we focus on understanding human attacker behaviors in order to optimize the defender's strategy. To achieve this goal, we model attacker-defender engagements as Markov Games and search for their Bayesian Stackelberg Equilibri…
Interactions among individuals in natural populations often occur in a dynamically changing environment. Understanding the role of environmental variation in population dynamics has long been a central topic in theoretical ecology and population biology. However, the key question of how individuals, in the middle of challenging social dilemmas (e.g., the "tragedy of the commons"), modulate their behaviors to adapt to the fluctuation of the environment has not yet been addressed satisfactorily. Utilizing evolutionary game theory and stochastic games, we develop a game-theoretical framework that…
Considering feedback of collective actions of cooperation on common resources has vital importance to reach sustainability. But such efforts may have not immediate consequence on the state of environment and it is unclear how they influence the strategic and environmental dynamics with feedbacks. To address this issue, we construct a feedback-evolving game model in which we consider the growth capacity of resources and the punishment efficiency on defectors who do not provide returns to the environment. Importantly, we further assume a delay in adopting the contribution of cooperative individu…
