arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新，休息一下吧，周末愉快~~
arXiv 2005.11853math.OCcs.LGcs.MA
收藏

随机Stackelberg安全博弈的无模型强化学习

Model-free Reinforcement Learning for Stochastic Stackelberg Security Games

Rajesh K Mishra, Deepanshu Vasal, Sriram Vishwanath

阅读 PDF
arXiv 页面 ↗HTML 原文 ↗

摘要：

本文考虑了两个局中人的序列随机Stackelberg对策，其中一个局中人是领导者，另一个局中人是跟随者。跟随者可以访问系统的状态，而领导者则不能。假设参与者按照各自的最佳利益行事，追随者的策略是对领导者的策略做出最佳反应。在这种情况下，领导者的优势是承诺一个政策，使自己的回报最大化的知识，追随者将发挥其政策的最佳反应。因此，两个参与者都收敛到一对策略，形成了博弈的斯塔克伯格均衡。最近，[1]提出了一种序列分解算法来计算这类博弈的Stackelberg均衡，它允许在线性时间内计算马尔可夫均衡策略，而不是像以前那样，在双指数时间内计算。在本文中，我们的想法扩展到一个MDP的动态是不知道的球员，提出了一个RL算法的基础上预期Sarsa学习Stackelberg均衡策略，通过模拟模型的MDP。我们使用粒子滤波器来估计一个共同的代理计算的最优策略的基础上，这是共同的两个球员的信念更新。我们提出了一个安全游戏的例子来说明我们的算法学习的政策。通过模拟MDP的模型。我们使用粒子滤波器来估计一个共同的代理计算的最优策略的基础上，这是共同的两个球员的信念更新。我们提出了一个安全游戏的例子来说明我们的算法学习的政策。

英文摘要：

In this paper, we consider a sequential stochastic Stackelberg game with two players, a leader and a follower. The follower has access to the state of the system while the leader does not. Assuming that the players act in their respective best interests, the follower's strategy is to play the best response to the leader's strategy. In such a scenario, the leader has the advantage of committing to a policy which maximizes its own returns given the knowledge that the follower is going to play the best response to its policy. Thus, both players converge to a pair of policies that form the Stackelberg equilibrium of the game. Recently,~[1] provided a sequential decomposition algorithm to compute the Stackelberg equilibrium for such games which allow for the computation of Markovian equilibrium policies in linear time as opposed to double exponential, as before. In this paper, we extend the idea to an MDP whose dynamics are not known to the players, to propose an RL algorithm based on Expected Sarsa that learns the Stackelberg equilibrium policy by simulating a model of the MDP. We use particle filters to estimate the belief update for a common agent which computes the optimal policy based on the information which is common to both the players. We present a security game example to illustrate the policy learned by our algorithm. by simulating a model of the MDP. We use particle filters to estimate the belief update for a common agent which computes the optimal policy based on the information which is common to both the players. We present a security game example to illustrate the policy learned by our algorithm.

机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。

↑