arXiv 1811.024832018-11-06cs.MAcs.AIcs.LG

Deep Reinforcement Learning for Green Security Games with Real-Time Information

Peking University（北京大学）；Carnegie Mellon University（卡内基梅隆大学）；Stanford University（斯坦福大学）；University of California, Berkeley（加州大学伯克利分校）；World Wild Fund for Nature（世界自然基金会）；Microsoft Research（微软研究院）

Yufei Wang, Zheyuan Ryan Shi, Lantao Yu, Yi Wu, Rohit Singh, Lucas Joppa, Fei Fang

阅读 PDF ↗

arXiv 页面 ↗HTML 原文 ↗

英文摘要相关论文

英文摘要：

Green Security Games (GSGs) have been proposed and applied to optimize patrols conducted by law enforcement agencies in green security domains such as combating poaching, illegal logging and overfishing. However, real-time information such as footprints and agents' subsequent actions upon receiving the information, e.g., rangers following the footprints to chase the poacher, have been neglected in previous work. To fill the gap, we first propose a new game model GSG-I which augments GSGs with sequential movement and the vital element of real-time information. Second, we design a novel deep reinforcement learning-based algorithm, DeDOL, to compute a patrolling strategy that adapts to the real-time information against a best-responding attacker. DeDOL is built upon the double oracle framework and the policy-space response oracle, solving a restricted game and iteratively adding best response strategies to it through training deep Q-networks. Exploring the game structure, DeDOL uses domain-specific heuristic strategies as initial strategies and constructs several local modes for efficient and parallelized training. To our knowledge, this is the first attempt to use Deep Q-Learning for security games.

机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。

阅读原文

阅读 PDF ↗

arXiv 页面 ↗HTML 原文 ↗

本文内容

英文摘要相关论文

机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。

继续探索

相关论文

Robust Reinforcement Learning Under Minimax Regret for Green Security

Harvard University（哈佛大学）；Carnegie Mellon University（卡内基梅隆大学）

Green security domains feature defenders who plan patrols in the face of uncertainty about the adversarial behavior of poachers, illegal loggers, and illegal fishers. Importantly, the deterrence effect of patrols on adversaries' future behavior makes patrol planning a sequential decision-making problem. Therefore, we focus on robust sequential patrol planning for green security following the minimax regret criterion, which has not been considered in the literature. We formulate the problem as a game between the defender and nature who controls the parameter values of the adversarial behavior a…

2021-06-15cs.AIcs.LGcs.MA

Interpretable Deep Reinforcement Learning for Green Security Games with Real-Time Information

University of Maryland（马里兰大学）

Green Security Games with real-time information (GSG-I) add the real-time information about the agents' movement to the typical GSG formulation. Prior works on GSG-I have used deep reinforcement learning (DRL) to learn the best policy for the agent in such an environment without any need to store the huge number of state representations for GSG-I. However, the decision-making process of DRL methods is largely opaque, which results in a lack of trust in their predictions. To tackle this issue, we present an interpretable DRL method for GSG-I that generates visualization to explain the decisions…

2022-11-09cs.AIcs.LG

Targets in Reinforcement Learning to solve Stackelberg Security Games

University of Maryland（马里兰大学）；Montgomery Blair High School（蒙哥马利布莱尔高中）；Wootton High School（伍顿高中）

Reinforcement Learning (RL) algorithms have been successfully applied to real world situations like illegal smuggling, poaching, deforestation, climate change, airport security, etc. These scenarios can be framed as Stackelberg security games (SSGs) where defenders and attackers compete to control target resources. The algorithm's competency is assessed by which agent is controlling the targets. This review investigates modeling of SSGs in RL with a focus on possible improvements of target representations in RL algorithms.

2022-11-30cs.AIcs.GTcs.LG

Deep Reinforcement Learning for FlipIt Security Game

Boston University（波士顿大学）；Center for Brains, Minds and Machines, MIT（麻省理工学院脑、心智与机器中心）；CMSA, Harvard University（哈佛大学数学与应用中心）

Reinforcement learning has shown much success in games such as chess, backgammon and Go. However, in most of these games, agents have full knowledge of the environment at all times. In this paper, we describe a deep learning model in which agents successfully adapt to different classes of opponents and learn the optimal counter-strategy using reinforcement learning in a game under partial observability. We apply our model to FlipIt, a two-player security game in which both players, the attacker and the defender, compete for ownership of a shared resource and only receive information on the cur…

2022-04-02cs.AIcs.GTcs.LG

NSGZero: Efficiently Learning Non-Exploitable Policy in Large-Scale Network Security Games with Neural Monte Carlo Tree Search

Nanyang Technological University（南洋理工大学）

How resources are deployed to secure critical targets in networks can be modelled by Network Security Games (NSGs). While recent advances in deep learning (DL) provide a powerful approach to dealing with large-scale NSGs, DL methods such as NSG-NFSP suffer from the problem of data inefficiency. Furthermore, due to centralized control, they cannot scale to scenarios with a large number of resources. In this paper, we propose a novel DL-based method, NSGZero, to learn a non-exploitable policy in NSGs. NSGZero improves data efficiency by performing planning with neural Monte Carlo Tree Search (MC…

2022-01-17cs.AIcs.CRcs.LG

Model-free Reinforcement Learning for Stochastic Stackelberg Security Games

In this paper, we consider a sequential stochastic Stackelberg game with two players, a leader and a follower. The follower has access to the state of the system while the leader does not. Assuming that the players act in their respective best interests, the follower's strategy is to play the best response to the leader's strategy. In such a scenario, the leader has the advantage of committing to a policy which maximizes its own returns given the knowledge that the follower is going to play the best response to its policy. Thus, both players converge to a pair of policies that form the Stackel…

2020-05-24cs.LGcs.MAmath.OC

更多相关论文收起更多论文

QFlip: An Adaptive Reinforcement Learning Strategy for the FlipIt Security Game

Khoury College of Computer Sciences, Northeastern University（东北大学库里计算机科学学院）

A rise in Advanced Persistent Threats (APTs) has introduced a need for robustness against long-running, stealthy attacks which circumvent existing cryptographic security guarantees. FlipIt is a security game that models attacker-defender interactions in advanced scenarios such as APTs. Previous work analyzed extensively non-adaptive strategies in FlipIt, but adaptive strategies rise naturally in practical interactions as players receive feedback during the game. We model the FlipIt game as a Markov Decision Process and introduce QFlip, an adaptive strategy for FlipIt based on temporal differen…

2019-12-21cs.CRcs.GTcs.LG

Learning Security Strategies through Game Play and Optimal Stopping

KTH Royal Institute of Technology（瑞典皇家理工学院）；KTH Center for Cyber Defense and Information Security（KTH网络防御与信息安全中心）

We study automated intrusion prevention using reinforcement learning. Following a novel approach, we formulate the interaction between an attacker and a defender as an optimal stopping game and let attack and defense strategies evolve through reinforcement learning and self-play. The game-theoretic perspective allows us to find defender strategies that are effective against dynamic attackers. The optimal stopping formulation gives us insight into the structure of optimal strategies, which we show to have threshold properties. To obtain the optimal defender strategies, we introduce T-FP, a fict…

2022-05-29cs.AIcs.CRcs.LG

6G感知安全：用于城市波束成形和攻击者检测的分布式博弈论强化学习

6G Sensing Security: Distributed Game-Theoretic RL for Urban Beamforming and Attacker Detection

研究6G ISAC系统中城市环境下主动攻击者的检测问题，采用博弈论方法建模合法用户与攻击者交互，集成到强化学习框架，有效应对了动态6G ISAC系统安全挑战。

2026-07-08cs.CRcs.GTcs.IT

双重任务巡逻：面向绿色安全的多臂老虎机

Dual-Mandate Patrols: Multi-Armed Bandits for Green Security

Harvard University（哈佛大学）；Carnegie Mellon University（卡内基梅隆大学）

本文将绿色安全巡逻形式化为随机多臂老虎机问题，提出利用奖励函数平滑性和动作可分解性的LIZARD算法，在保证收敛率的同时优化短期性能，并在柬埔寨真实偷猎数据上取得提升。

2024-04-26cs.LGstat.ML

求解大规模城市网络安全博弈的基于树的随机优化方法

Tree-Based Stochastic Optimization for Solving Large-Scale Urban Network Security Games

针对大规模城市网络安全博弈纳什均衡求解难题，本文提出基于树的随机优化框架TSO，结合树状动作表征与采样剪枝机制，实验证明其性能优于现有基线算法。

2025-11-13cs.LG

Designing the Game to Play: Optimizing Payoff Structure in Security Games

Carnegie Mellon University（卡内基梅隆大学）；University of Southampton（南安普顿大学）；World Wide Fund for Nature（世界自然基金会）；Swarthmore College（斯沃斯莫尔学院）

Effective game-theoretic modeling of defender-attacker behavior is becoming increasingly important. In many domains, the defender functions not only as a player but also the designer of the game's payoff structure. We study Stackelberg Security Games where the defender, in addition to allocating defensive resources to protect targets from the attacker, can strategically manipulate the attacker's payoff under budget constraints in weighted L^p-norm form regarding the amount of change. Focusing on problems with weighted L^1-norm form constraint, we present (i) a mixed integer linear program-base…

2018-05-22cs.AIcs.GTcs.MA

Solving Large-Scale Extensive-Form Network Security Games via Neural Fictitious Self-Play

Nanyang Technological University（南洋理工大学）；Dartmouth College（达特茅斯学院）

Securing networked infrastructures is important in the real world. The problem of deploying security resources to protect against an attacker in networked domains can be modeled as Network Security Games (NSGs). Unfortunately, existing approaches, including the deep learning-based approaches, are inefficient to solve large-scale extensive-form NSGs. In this paper, we propose a novel learning paradigm, NSG-NFSP, to solve large-scale extensive-form NSGs based on Neural Fictitious Self-Play (NFSP). Our main contributions include: i) reforming the best response (BR) policy network in NFSP to be a …

2021-06-02cs.AIcs.GTcs.LG

防御城市免受多无人机攻击：一种序贯斯塔克尔伯格安全博弈方法

Defending a City from Multi-Drone Attacks: A Sequential Stackelberg Security Games Approach

针对多无人机城市攻击，研究将其建模为序贯斯塔克尔伯格安全博弈，提出S2D2算法输出混合序贯防御策略，在80个真实城市数据上优于贪婪启发式方法，并证明其能输出近似强斯塔克尔伯格均衡（SSE）。

2025-08-15cs.MA