单边零和部分可观测随机对策的Stackelberg均衡
The Stackelberg Equilibrium for One-sided Zero-sum Partially Observable Stochastic Games
摘要:
将攻击者和防御者的网络安全问题描述为部分可观察的随机博弈已经成为一种趋势。其中,单边两人零和部分可观察随机博弈(OTZ-POSG)已经成为一个流行的模型,因为它允许玩家在系统的部分知识的基础上竞争多个阶段。OTZ-POSG的所有现有工作都集中在同时移动场景上,并假设一个玩家的动作在执行过程中是私有的。然而,这种假设可能会变得有问题,因为一个玩家的动作可能会被对手通过部署动作检测策略来检测。因此,在本文中,我们提出了一个回合为基础的OTZ-POSG与公共行动的假设,并研究了这个游戏的Stackelberg均衡的存在性和性质。我们首先证明了存在的Stackelberg平衡的一阶段的情况下,并表明,一阶段的游戏可以转换成一个线性分式规划问题,因此解决了线性规划。对于多个阶段,主要的挑战是信息泄漏问题,因为公共运行时动作向对手透露了某些私人信息,并允许对手在未来获得更多奖励。为了解决这个问题,我们引入了$ε$ -Stackelberg均衡的概念,并证明了有限时域OTZ-POSG可以达到这个均衡.我们提出了一个空间划分的方法来解决游戏迭代,并表明,价值函数的领导者是分段线性和价值函数的追随者是分段常数的多个阶段。最后,通过实例说明了空间划分方法,并证明了值函数是分段线性的和分段常数的。
英文摘要:
Formulating cyber-security problems with attackers and defenders as a partially observable stochastic game has become a trend recently. Among them, the one-sided two-player zero-sum partially observable stochastic game (OTZ-POSG) has emerged as a popular model because it allows players to compete for multiple stages based on partial knowledge of the system. All existing work on OTZ-POSG has focused on the simultaneous move scenario and assumed that one player's actions are private in the execution process. However, this assumption may become questionable since one player's action may be detected by the opponent through deploying action detection strategies. Hence, in this paper, we propose a turn-based OTZ-POSG with the assumption of public actions and investigate the existence and properties of a Stackelberg equilibrium for this game. We first prove the existence of the Stackelberg equilibrium for the one-stage case and show that the one-stage game can be converted into a linear-fractional programming problem and therefore solved by linear programming. For multiple stages, the main challenge is the information leakage issue as the public run-time action reveals certain private information to the opponent and allows the opponent to achieve more rewards in the future. To deal with this issue, we adopt the concept of $ε$-Stackelberg equilibrium and prove that this equilibrium can be achieved for finite-horizon OTZ-POSGs. We propose a space partition approach to solve the game iteratively and show that the value function of the leader is piece-wise linear and the value function of the follower is piece-wise constant for multiple stages. Finally, examples are given to illustrate the space partition approach and show that value functions are piece-wise linear and piece-wise constant.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。