In this work, we introduce decentralized network interdiction games, which model the interactions among multiple interdictors with differing objectives operating on a common network. As a starting point, we focus on decentralized shortest path interdiction (DSPI) games, where multiple interdictors try to increase the shortest path lengths of their own adversaries, who all attempt to traverse a common network. We first establish results regarding the existence of equilibria for DSPI games under both discrete and continuous interdiction strategies. To compute such an equilibrium, we present a reformulation of the DSPI games, which leads to a generalized Nash equilibrium problem (GNEP) with non-shared constraints. While such a problem is computationally challenging in general, we show that under continuous interdiction actions, a DSPI game can be formulated as a linear complementarity problem and solved by Lemke's algorithm. In addition, we present decentralized heuristic algorithms based on best response dynamics for games under both continuous and discrete interdiction strategies. Finally, we establish theoretical bounds on the worst-case efficiency loss of equilibria in DSPI games, with such loss caused by the lack of coordination among noncooperative interdictors, and use the decentralized algorithms to empirically study the average-case efficiency loss.
Many relevant applications from diverse areas such as marketing, wildlife conservation, or defending critical infrastructure can be modeled as interdiction games. In this work, we introduce interdiction games whose objective is a monotone and submodular set function. Given a ground set of items, the leader interdicts the usage of some of the items of the follower in order to minimize the objective value achievable by the follower, who seeks to maximize a submodular set function over the uninterdicted items subject to knapsack constraints. We propose an exact branch-and-cut algorithm for these …
This article poses the following problem: Does there exist a probability distribution over subsets of a finite partially ordered set (poset), such that a set of constraints involving marginal probabilities of the poset's elements and maximal chains is satisfied? We present a combinatorial algorithm to positively resolve this question. The algorithm can be implemented in polynomial time in the special case where maximal chain probabilities are affine functions of their elements. This existence problem is relevant for the equilibrium characterization of a generic strategic interdiction game on a…
Conditional Value at Risk (CVaR) is widely used to account for the preferences of a risk-averse agent in the extreme loss scenarios. To study the effectiveness of randomization in interdiction games with an interdictor that is both risk and ambiguity averse, we introduce a distributionally robust network interdiction game where the interdictor randomizes over the feasible interdiction plans in order to minimize the worst-case CVaR of the flow with respect to both the unknown distribution of the capacity of the arcs and his mixed strategy over interdicted arcs. The flow player, on the contrary,…
This work studies Stackelberg network interdiction games -- an important class of games in which a defender first allocates (randomized) defense resources to a set of critical nodes on a graph while an adversary chooses its path to attack these nodes accordingly. We consider a boundedly rational adversary in which the adversary's response model is based on a dynamic form of classic logit-based discrete choice models. We show that the problem of finding an optimal interdiction strategy for the defender in the rational setting is NP-hard. The resulting optimization is in fact non-convex and addi…
Randomized Max-Vertex-Coverage Interdiction under Matroid Constraints
Academy of Mathematics and Systems Science, Chinese Academy of Sciences（中国科学院数学与系统科学研究院）；Beijing Normal University-Zhuhai（北京师范大学珠海校区）；Beijing Normal-Hong Kong Baptist University（北京浸会大学）
This paper studies an asymptotic solvability problem for linear quadratic (LQ) mean field games with controlled diffusions and indefinite weights for the state and control in the costs. We employ a rescaling approach to derive a low dimensional Riccati ordinary differential equation (ODE) system, which characterizes a necessary and sufficient condition for asymptotic solvability. The rescaling technique is further used for performance estimates, establishing an $O(1/N)$-Nash equilibrium for the obtained decentralized strategies.
This paper is concerned with a new class of mean-field games which involve a finite number of agents. Necessary and sufficient conditions are obtained for the existence of the decentralized open-loop Nash equilibrium in terms of non-standard forward-backward stochastic differential equations (FBSDEs). By solving the FBSDEs, we design a set of decentralized strategies by virtue of two differential Riccati equations. Instead of the $\varepsilon$-Nash equilibrium in classical mean-field games, the set of decentralized strategies is shown to be a Nash equilibrium. For the infinite-horizon problem,…
We address the generalized Nash equilibrium seeking problem for a population of agents playing aggregative games with affine coupling constraints. We focus on semi-decentralized communication architectures, where there is a central coordinator able to gather and broadcast signals of aggregative nature to the agents. By exploiting the framework of monotone operator theory and operator splitting, we first critically review the most relevant available algorithms and then design two novel schemes: (i) a single-layer, fixed-step algorithm with convergence guarantee for general (non cocoercive, non-…
Decentralized online learning for seeking generalized Nash equilibrium (GNE) of noncooperative games in dynamic environments is studied in this paper. Each player aims at selfishly minimizing its own time-varying cost function subject to time-varying coupled constraints and local feasible set constraints. Only local cost functions and local constraints are available to individual players, who can receive their neighbors' information through a fixed and connected graph. In addition, players have no prior knowledge of cost functions and local constraint functions in the future time. In this sett…
Increasing connectivity of communication networks enables large-scale distributed processing over networks and improves the efficiency for information exchange. However, malware and virus can take advantage of the high connectivity to spread over the network and take control of devices and servers for illicit purposes. In this paper, we use an SIS epidemic model to capture the virus spreading process and develop a virus-resistant weight adaptation scheme to mitigate the spreading over the network. We propose a differential game framework to provide a theoretic underpinning for decentralized mi…
This paper studies distributed online bandit learning of generalized Nash equilibria for online game, where cost functions of all players and coupled constraints are time-varying. The values rather than full information of cost and local constraint functions are revealed to local players gradually. The goal of each player is to selfishly minimize its own cost function with no future information subject to a strategy set constraint and time-varying coupled inequality constraints. To this end, a distributed online algorithm based on mirror descent and one-point bandit feedback is designed for se…
The value of cooperation in pursuit-evasion games is investigated. The considered setting is that of three pursuers chasing one evader in a planar environment. The optimal evader trajectory for a well-known decentralized pursuer strategy is characterized. This result is instrumental to derive upper and lower bounds to the game length, in the case in which the pursuers cooperate in the chasing strategy. It is shown that the cooperation cannot reduce the capture time by more than one half with respect to the decentralized case, and that such bound is tight.
This paper studies mean field games for multi-agent systems with control-dependent multiplicative noises. For the general systems with nonuniform agents, we obtain a set of decentralized strategies by solving an auxiliary limiting optimal control problem subject to consistent mean field approximations. The set of decentralized strategies is further shown to be an $\varepsilon$-Nash equilibrium. For the integrator multiagent systems, we design a set of $\varepsilon$-Nash strategies by exploiting the convexity property of the limiting problem. It is shown that under the mild conditions all the a…
Softmax policy gradient is a popular algorithm for policy optimization in single-agent reinforcement learning, particularly since projection is not needed for each gradient update. However, in multi-agent systems, the lack of central coordination introduces significant additional difficulties in the convergence analysis. Even for a stochastic game with identical interest, there can be multiple Nash Equilibria (NEs), which disables proof techniques that rely on the existence of a unique global optimum. Moreover, the softmax parameterization introduces non-NE policies with zero gradient, making …
We study the convergence properties of decentralized fictitious play (DFP) for the class of near-potential games where the incentives of agents are nearly aligned with a potential function. In DFP, agents share information only with their current neighbors in a sequence of time-varying networks, keep estimates of other agents' empirical frequencies, and take actions to maximize their expected utility functions computed with respect to the estimated empirical frequencies. We show that empirical frequencies of actions converge to a set of strategies with potential function values that are larger…
This paper studies asymptotic solvability of a linear quadratic (LQ) mean field social optimization problem with controlled diffusions and indefinite state and control weights. Starting with an $N$-agent model, we employ a rescaling approach to derive a low-dimensional Riccati ordinary differential equation (ODE) system, which characterizes a necessary and sufficient condition for asymptotic solvability. The decentralized control obtained from the mean field limit ensures a bounded optimality loss in minimizing the social cost having magnitude $O(N)$, which implies an optimality loss of $O(1/N…
Centralized Versus Decentralized Team Games of Distributed Stochastic Differential Decision Systems with Noiseless Information Structures-Part II: Applications
University of Cyprus（塞浦路斯大学）；University of Ottawa（渥太华大学）
Convergence of Decentralized Actor-Critic Algorithm in General-sum Markov Games
Electrical Engineering and Computer Sciences (EECS) at UC Berkeley（加州大学伯克利分校电子工程与计算机科学系）；Operations Research and Information Engineering (ORIE) at Cornell University（康奈尔大学运筹学与信息工程系）
High-Probability Nash Regret for Decentralized Learning in Markov $α$-Potential Games: Episodic and Fully Online Asynchronous Algorithms with Applications to Markov Congestion Games
University of Illinois Urbana-Champaign（伊利诺伊大学厄巴纳-香槟分校）
Centralized Versus Decentralized Team Games of Distributed Stochastic Differential Decision Systems with Noiseless Information Structures-Part I: General Theory
This paper studies social optima and Nash games for mean field linear quadratic control systems, where subsystems are coupled via dynamics and individual costs. For the social control problem, we first obtain a set of forward-backward stochastic differential equations (FBSDE) from variational analysis, and construct a feedback-type control by decoupling the FBSDE. By using solutions of two Riccati equations, we design a set of decentralized control laws, which is further proved to be asymptotically social optimal. Two equivalent conditions are given for uniform stabilization of the systems in …
We consider a class of concave continuous games in which the corresponding admissible strategy profile of each player underlies affine coupling constraints. We propose a novel algorithm that leads the relevant population dynamic toward Nash equilibrium. This algorithm is based on a mirror ascent algorithm, which suits with the framework of no-regret online learning, and on the augmented Lagrangian method. The decentralization aspect of the algorithm corresponds to the aspects that the iterate of each player requires the local information about how she contributes to the coupling constraints an…
Splunk（斯普伦克）；University of Southern California（南加州大学）；University of Florida（佛罗里达大学）
Min-max saddle point games have recently been intensely studied, due to their wide range of applications, including training Generative Adversarial Networks (GANs). However, most of the recent efforts for solving them are limited to special regimes such as convex-concave games. Further, it is customarily assumed that the underlying optimization problem is solved either by a single machine or in the case of multiple machines connected in centralized fashion, wherein each one communicates with a central node. The latter approach becomes challenging, when the underlying communications network has…
The paper deals with Age of Information (AoI) in a network of multiple sources and parallel queues with buffering capabilities, preemption in service and losses in served packets. The queues do not communicate between each other and the packets are dispatched through the queues according to a predefined probabilistic routing. By making use of the Stochastic Hybrid System (SHS) method, we provide a derivation of the average AoI of a system of two parallel queues (with and without buffer capabilities) and compare the results with those of a single queue. We show that known results of packets del…
We consider the control of decentralized learning dynamics for agents in an anti-coordination network game. In the anti-coordination network game, there is a preferred action in the absence of neighbors' actions, and the utility an agent receives from the preferred action decreases as more of its neighbors select the preferred action, potentially causing the agent to select a less desirable action. The decentralized dynamics that is based on the iterated elimination of dominated strategies converge for the considered game. Given a convergent action profile, we measure anti-coordination by the …
Multiple autonomous agents interact over a random communication network to maximize their individual utility functions which depend on the actions of other agents. We consider decentralized best-response with inertia type algorithms in which agents form beliefs about the future actions of other players based on local information, and take an action that maximizes their expected utility computed with respect to these beliefs or continue to take their previous action. We show convergence of these types of algorithms to a Nash equilibrium in weakly acyclic games under the condition that the belie…
