Reinforcement learning (RL) has been applied to attack graphs for penetration testing, however, trained agents do not reflect reality because the attack graphs lack operational nuances typically captured within the intelligence preparation of the battlefield (IPB) that include notions of (cyber) terrain. In particular, current practice constructs attack graphs exclusively using the Common Vulnerability Scoring System (CVSS) and its components. We present methods for constructing attack graphs using notions from IPB on cyber terrain analysis of obstacles, avenues of approach, key terrain, observation and fields of fire, and cover and concealment. We demonstrate our methods on an example where firewalls are treated as obstacles and represented in (1) the reward space and (2) the state dynamics. We show that terrain analysis can be used to bring realism to attack graphs for RL.
National Security Institute, Virginia Polytechnic University（弗吉尼亚理工大学国家安全研究所）；Deloitte & Touche LLP（德勤华永会计师事务所）
Reinforcement learning (RL) operating on attack graphs leveraging cyber terrain principles are used to develop reward and state associated with determination of surveillance detection routes (SDR). This work extends previous efforts on developing RL methods for path analysis within enterprise networks. This work focuses on building SDR where the routes focus on exploring the network services while trying to evade risk. RL is utilized to support the development of these routes by building a reward mechanism that would help in realization of these paths. The RL algorithm is modified to have a no…
Deloitte Consulting, LLC（德勤咨询有限责任公司）；Hume Center for National Security and Technology, Virginia Polytechnic University（弗吉尼亚理工大学休姆国家安全与技术中心）；Deloitte & Touche LLP（德勤会计师事务所）
Cyber attacks pose existential threats to nations and enterprises. Current practice favors piece-wise analysis using threat-models in the stead of rigorous cyber terrain analysis and intelligence preparation of the battlefield. Automated penetration testing using reinforcement learning offers a new and promising approach for developing methodologies that are driven by network structure and cyber terrain, that can be later interpreted in terms of threat-models, but that are principally network-driven analyses. This paper presents a novel method for crown jewel analysis termed CJA-RL that uses r…
University of Technology Sydney（悉尼科技大学）；Defence Science and Technology Group（国防科技集团）
Penetration testing the organised attack of a computer system in order to test existing defences has been used extensively to evaluate network security. This is a time consuming process and requires in-depth knowledge for the establishment of a strategy that resembles a real cyber-attack. This paper presents a novel deep reinforcement learning architecture with hierarchically structured agents called HA-DRL, which employs an algebraic action decomposition strategy to address the large discrete action space of an autonomous penetration testing simulator where the number of actions is exponentia…
Penetration testing (pentesting) involves performing a controlled attack on a computer system in order to assess it's security. Although an effective method for testing security, pentesting requires highly skilled practitioners and currently there is a growing shortage of skilled cyber security professionals. One avenue for alleviating this problem is automate the pentesting process using artificial intelligence techniques. Current approaches to automated pentesting have relied on model-based planning, however the cyber security landscape is rapidly changing making maintaining up-to-date model…
Learning Robust Penetration Testing Policies under Partial Observability: A systematic evaluation
Cyber Defence Lab, CISS Department Royal Military Academy（国防网络安全实验室，信息与系统科学系皇家军事学院）；AI Lab, Department of Computer Science Vrije Universiteit Brussel（人工智能实验室，计算机科学系自由大学布鲁塞尔）
National Security Institute, Virginia Tech（弗吉尼亚理工大学国家安全研究所）；Deloitte & Touche LLP（德勤华永会计师事务所）；Deloitte Consulting LLP（德勤咨询有限公司）
Reinforcement learning (RL), in conjunction with attack graphs and cyber terrain, are used to develop reward and state associated with determination of optimal paths for exfiltration of data in enterprise networks. This work builds on previous crown jewels (CJ) identification that focused on the target goal of computing optimal paths that adversaries may traverse toward compromising CJs or hosts within their proximity. This work inverts the previous CJ approach based on the assumption that data has been stolen and now must be quietly exfiltrated from the network. RL is utilized to support the …
Pentest-R1: Towards Autonomous Penetration Testing Reasoning Optimized via Two-Stage Reinforcement Learning
State Key Laboratory of Cyberspace Security Defense, Institute of Information Engineering, Chinese Academy of Sciences（中国科学院网络空间安全防御重点实验室，信息工程研究所）；School of Cyber Security, University of Chinese Academy of Sciences（中国科学院大学网络与安全学院）
Guided Reasoning in LLM-Driven Penetration Testing Using Structured Attack Trees
Department of Electrical and Computer Engineering, Rochester Institute of Technology（罗切斯特理工学院电子与计算机工程系）；Institute for Informatics and Applied Technology, Gonzaga University（贡萨加大学信息学与应用技术研究所）
This thesis develops data-driven machine learning algorithms to managing and optimizing the next-generation highly complex cyberphysical systems, which desperately need ground-breaking control, monitoring, and decision making schemes that can guarantee robustness, scalability, and situational awareness. The present thesis first develops principled methods to make generic machine learning models robust against distributional uncertainties and adversarial data. Particular focus will be on parametric models where some training data are being used to learn a parametric model. The developed framewo…
A honeynet is a promising active cyber defense mechanism. It reveals the fundamental Indicators of Compromise (IoCs) by luring attackers to conduct adversarial behaviors in a controlled and monitored environment. The active interaction at the honeynet brings a high reward but also introduces high implementation costs and risks of adversarial honeynet exploitation. In this work, we apply infinite-horizon Semi-Markov Decision Process (SMDP) to characterize a stochastic transition and sojourn time of attackers in the honeynet and quantify the reward-risk trade-off. In particular, we design adapti…
Lane Department of Computer Science and Electrical Engineering, West Virginia University（计算机科学与电气工程系，西弗吉尼亚大学）
Despite the relentless progress of deep learning models in analyzing the system conditions under cyber-physical events, their abilities are limited in the power system domain due to data availability issues, cost of data acquisition, and lack of interpretation and extrapolation for the data beyond the training windows. In addition, the integration of distributed energy resources (DERs) such as wind and solar generations increases the complexities and nonlinear nature of power systems. Therefore, an interpretable and reliable methodology is of utmost need to increase the confidence of power sys…
Delft University of Technology（代尔夫特理工大学）；Rochester Institute of Technology（罗切斯特理工大学）
Attack graphs (AG) are used to assess pathways availed by cyber adversaries to penetrate a network. State-of-the-art approaches for AG generation focus mostly on deriving dependencies between system vulnerabilities based on network scans and expert knowledge. In real-world operations however, it is costly and ineffective to rely on constant vulnerability scanning and expert-crafted AGs. We propose to automatically learn AGs based on actions observed through intrusion alerts, without prior expert knowledge. Specifically, we develop an unsupervised sequence learning system, SAGE, that leverages …
