With the increasing system complexity and attack sophistication, the necessity of autonomous cyber defense becomes vivid for cyber and cyber-physical systems (CPSs). Many existing frameworks in the current state-of-the-art either rely on static models with unrealistic assumptions, or fail to satisfy the system safety and security requirements. In this paper, we present a new hybrid autonomous agent architecture that aims to optimize and verify defense policies of reinforcement learning (RL) by incorporating constraints verification (using satisfiability modulo theory (SMT)) into the agent's decision loop. The incorporation of SMT does not only ensure the satisfiability of safety and security requirements, but also provides constant feedback to steer the RL decision-making toward safe and effective actions. This approach is critically needed for CPSs that exhibit high risk due to safety or security violations. Our evaluation of the presented approach in a simulated CPS environment shows that the agent learns the optimal policy fast and defeats diversified attack strategies in 99\% cases.
Naval Information Warfare Center Pacific（太平洋海军信息战中心）；Laboratory for Advanced Cybersecurity Research（高级网络安全研究实验室）
Deceptive elements, including honeypots and decoys, were incorporated into the Microsoft CyberBattleSim experimentation and research platform. The defensive capabilities of the deceptive elements were tested using reinforcement learning based attackers in the provided capture the flag environment. The attacker's progress was found to be dependent on the number and location of the deceptive elements. This is a promising step toward reproducibly testing attack and defense algorithms in a simulated enterprise network with deceptive defensive elements.
Command and Control Engineering College, Army Engineering University of PLA（中国人民解放军陆军工程大学指挥控制工程学院）；troop of PLA（中国人民解放军31436部队）；Academy of Military Science（军事科学院）
The existing network attack and defense method can be regarded as game, but most of the game only involves network domain, not multiple domain cyberspace. To address this challenge, this paper proposed a multiple domain cyberspace attack and defense game model based on reinforcement learning. We define the multiple domain cyberspace include physical domain, network domain and digital domain. By establishing two agents, representing the attacker and the defender respectively, defender will select the multiple domain actions in the multiple domain cyberspace to obtain defender's optimal reward b…
Development of autonomous cyber system defense strategies and action recommendations in the real-world is challenging, and includes characterizing system state uncertainties and attack-defense dynamics. We propose a data-driven deep reinforcement learning (DRL) framework to learn proactive, context-aware, defense countermeasures that dynamically adapt to evolving adversarial behaviors while minimizing loss of cyber system operations. A dynamic defense optimization problem is formulated with multiple protective postures against different types of adversaries with varying levels of skill and per…
Machine Learning-Based Cyber Defense for Cloud Infrastructure: An Adaptive Deep Q-Network Architecture for Intelligent Intrusion Detection and Automated Threat Mitigation
Wilmington University（威尔明顿大学）；Washington University of Science and Technology（华盛顿科技大学）；South Dakota School of Mines & Technology（南达科他矿业理工学院）；South Dakota State University（南达科他州立大学）；St. Francis College（圣弗朗西斯学院）；Stanton University（斯坦顿大学）
Naval Information Warfare Center Pacific（太平洋海军信息战中心）；Laboratory for Advanced Cybersecurity Research（高级网络安全研究实验室）
The performance of artificial intelligence (AI) algorithms in practice depends on the realism and correctness of the data, models, and feedback (labels or rewards) provided to the algorithm. This paper discusses methods for improving the realism and ecological validity of AI used for autonomous cyber defense by exploring the potential to use Inverse Reinforcement Learning (IRL) to gain insight into attacker actions, utilities of those actions, and ultimately decision points which cyber deception could thwart. The Tularosa study, as one example, provides experimental data of real-world techniqu…
CyberArena: Benchmarking AI Agents in Live Symmetric Attack-and-Defense Combat
AI Open Source Foundation（2077AI开源基金会）；UC Santa Cruz（加州大学圣克鲁兹分校）；UC Berkeley（加州大学伯克利分校）；UC Santa Barbara（加州大学圣巴巴拉分校）；University of Wisconsin–Madison（威斯康星大学麦迪逊分校）；Massachusetts Institute of Technology（麻省理工学院）；University of Toronto（多伦多大学）；Johns Hopkins University（约翰斯·霍普金斯大学）；Carnegie Mellon University（卡内基梅隆大学）；Purdue University（普渡大学）；Texas A&M University（德克萨斯农工大学）
The scale of Internet-connected systems has increased considerably, and these systems are being exposed to cyber attacks more than ever. The complexity and dynamics of cyber attacks require protecting mechanisms to be responsive, adaptive, and scalable. Machine learning, or more specifically deep reinforcement learning (DRL), methods have been proposed widely to address these issues. By incorporating deep learning into traditional RL, DRL is highly capable of solving complex, dynamic, and especially high-dimensional cyber defense problems. This paper presents a survey of DRL approaches develop…
Nowadays, considering the speed of the processes and the amount of data used in cyber defense, it cannot be expected to have an effective defense by using only human power without the help of automation systems. However, for the effective defense against dynamically evolving attacks on networks, it is difficult to develop software with conventional fixed algorithms. This can be achieved by using artificial intelligence methods that provide flexibility and learning capability. The likelihood of developing cyber defense capabilities through increased intelligence of defense systems is quite high…
The 1st International Workshop on Adaptive Cyber Defense was held as part of the 2021 International Joint Conference on Artificial Intelligence. This workshop was organized to share research that explores unique applications of Artificial Intelligence (AI) and Machine Learning (ML) as foundational capabilities for the pursuit of adaptive cyber defense. The cyber domain cannot currently be reliably and effectively defended without extensive reliance on human experts. Skilled cyber defenders are in short supply and often cannot respond fast enough to cyber threats. Building on recent advances in…
From denial-of-service attacks to spreading of ransomware or other malware across an organization's network, it is possible that manually operated defenses are not able to respond in real time at the scale required, and when a breach is detected and remediated the damage is already made. Autonomous cyber defenses therefore become essential to mitigate the risk of successful attacks and their damage, especially when the response time, effort and accuracy required in those defenses is impractical or impossible through defenses operated exclusively by humans. Autonomous agents have the potential …
With the recognition of cyberspace as an operating domain, concerted effort is now being placed on addressing it in the whole-of-domain manner found in land, sea, undersea, air, and space domains. Among the first steps in this effort is applying the standard supporting concepts of security, defense, and deterrence to the cyber domain. This paper presents an architecture that helps realize forward defense in cyberspace, wherein adversarial actions are repulsed as close to the origin as possible. However, substantial work remains in making the architecture an operational reality including furthe…
Adversarial dynamics are a critical facet within the cyber security domain, in which there exists a co-evolution between attackers and defenders in any given threat scenario. While defenders leverage capabilities to minimize the potential impact of an attack, the adversary is simultaneously developing countermeasures to the observed defenses. In this study, we develop a set of tools to model the adaptive strategy formulation of an intelligent actor against an active cyber defensive system. We encode strategies as binary chromosomes representing finite state machines that evolve according to Ho…
A model of strategy formulation is used to study how an adaptive attacker learns to overcome a moving target cyber defense. The attacker-defender interaction is modeled as a game in which a defender deploys a temporal platform migration defense. Against this defense, a population of attackers develop strategies specifying the temporal ordering of resource investments that bring targeted zero-day exploits into existence. Attacker response to two defender temporal platform migration scheduling policies are examined. In the first defender scheduling policy, the defender selects the active platfor…
In this chapter, we present an approach using formal methods to synthesize reactive defense strategy in a cyber network, equipped with a set of decoy systems. We first generalize formal graphical security models--attack graphs--to incorporate defender's countermeasures in a game-theoretic model, called an attack-defend game on graph. This game captures the dynamic interactions between the defender and the attacker and their defense/attack objectives in formal logic. Then, we introduce a class of hypergames to model asymmetric information created by decoys in the attacker-defender interactions.…
This paper sets the context for the urgency for cyber autonomy, and the current gaps of the cyber security industry. A novel framework proposing four phases of maturity for full cyber autonomy will be discussed. The paper also reviews new and emerging cyber security automation techniques and tools, and discusses their impact on society, the perceived cyber security skills gap/shortage and national security. We will also be discussing the delicate balance between national security, human rights and ethics, and the potential demise of the manual penetration testing industry in the face of automa…
The concept of active cyber defense has been proposed for years. However, there are no mathematical models for characterizing the effectiveness of active cyber defense. In this paper, we fill the void by proposing a novel Markov process model that is native to the interaction between cyber attack and active cyber defense. Unfortunately, the native Markov process model cannot be tackled by the techniques we are aware of. We therefore simplify, via mean-field approximation, the Markov process model as a Dynamic System model that is amenable to analysis. This allows us to derive a set of valuable…
Social engineering attacks are phenomena that are equally applicable to both the physical world and cyberspace. These attacks in the physical world have been studied for a much longer time than their counterpart in cyberspace. This motivates us to investigate how social engineering attacks in the physical world and cyberspace relate to each other, including their common characteristics and unique features. For this purpose, we propose a methodology to unify social engineering attacks and defenses in the physical world and cyberspace into a single framework, including: (i) a systematic model ba…
The Advanced Metering Infrastructure (AMI) is a vital element in the current development of the smart grid. AMI technologies provide electric utilities with an effective way of continuous monitoring and remote control of smart grid components. However, owing to its increasing scale and cyber-physical nature, the AMI has been faced with security threats in both cyber and physical domains. This paper provides a comprehensive review of the crucial cyber-physical attacks and counter defense mechanisms in the AMI. First, two attack surfaces are surveyed in the AMI including the communication networ…
Cybersecurity dynamics is a mathematical approach to modeling and analyzing cyber attack-defense interactions in networks. In this paper, we advance the state-of-the-art in characterizing one kind of cybersecurity dynamics, known as preventive and reactive cyber defense dynamics, which is a family of highly nonlinear system models. We prove that this dynamics in its general form with time-dependent parameters is globally attractive when the time-dependent parameters are ergodic, and is (almost) periodic when the time-dependent parameters have the stronger properties of being (almost) periodic.…
Within the future Global Information Grid, complex massively interconnected systems, isolated defense vehicles, sensors and effectors, and infrastructures and systems demanding extremely low failure rates, to which human security operators cannot have an easy access and cannot deliver fast enough reactions to cyber-attacks, need an active, autonomous and intelligent cyber defense. Multi Agent Systems for Cyber Defense may provide an answer to this requirement. This paper presents the concept and architecture of an Autonomous Intelligent Cyber defense Agent (AICA). First, we describe the ration…
Advanced Persistent Threats (APTs) have recently emerged as a significant security challenge for a cyber-physical system due to their stealthy, dynamic and adaptive nature. Proactive dynamic defenses provide a strategic and holistic security mechanism to increase the costs of attacks and mitigate the risks. This work proposes a dynamic game framework to model a long-term interaction between a stealthy attacker and a proactive defender. The stealthy and deceptive behaviors are captured by the multi-stage game of incomplete information, where each player has his own private information unknown t…
This report - a major revision of its previous release - describes a reference architecture for intelligent software agents performing active, largely autonomous cyber-defense actions on military networks of computing and communicating devices. The report is produced by the North Atlantic Treaty Organization (NATO) Research Task Group (RTG) IST-152 "Intelligent Autonomous Agents for Cyber Defense and Resilience". In a conflict with a technically sophisticated adversary, NATO military tactical networks will operate in a heavily contested battlefield. Enemy software cyber agents - malware - will…
In the coming years, the future of military combat will include, on one hand, artificial intelligence-optimized complex command, control, communications, computers, intelligence, surveillance and reconnaissance (C4ISR) and networks and, on the other hand, autonomous intelligent Things fighting autonomous intelligent Things at a fast pace. Under this perspective, enemy forces will seek to disable or disturb our autonomous Things and our complex infrastructures and systems. Autonomy, scale and complexity in our defense systems will trigger new cyber-attack strategies, and autonomous intelligent …
University of Maryland, Baltimore County（马里兰大学巴尔的摩郡分校）；University of North Carolina, Wilmington（北卡罗来纳大学威尔明顿分校）
Cyber-defense systems are being developed to automatically ingest Cyber Threat Intelligence (CTI) that contains semi-structured data and/or text to populate knowledge graphs. A potential risk is that fake CTI can be generated and spread through Open-Source Intelligence (OSINT) communities or on the Web to effect a data poisoning attack on these systems. Adversaries can use fake CTI examples as training input to subvert cyber defense systems, forcing the model to learn incorrect inputs to serve their malicious needs. In this paper, we automatically generate fake CTI text descriptions using tran…
It is challenging for a security analyst to detect or defend against cyber-attacks. Moreover, traditional defense deployment methods require the security analyst to manually enforce the defenses in the presence of uncertainties about the defense to deploy. As a result, it is essential to develop an automated and resilient defense deployment mechanism to thwart the new generation of attacks. In this paper, we propose a framework based on Markov Decision Process (MDP) and Q-learning to automatically generate optimal defense solutions for networked system states. The framework consists of four ph…
COVID-19 (Coronavirus) hit the global society and economy with a big surprise. In particular, work-from-home has become a new norm for employees. Despite the fact that COVID-19 can equally attack innocent people and cybercriminals, it is ironic to see surges in cyberattacks leveraging COVID-19 as a theme, dubbed COVID-19 themed cyberattacks or COVID-19 attacks for short, which represent a new phenomenon that has yet to be systematically understood. In this paper, we make the first step towards fully characterizing the landscape of these attacks, including their sophistication via the Cyber Kil…
Deception techniques have been widely seen as a game changer in cyber defense. In this paper, we review representative techniques in honeypots, honeytokens, and moving target defense, spanning from the late 1980s to the year 2021. Techniques from these three domains complement with each other and may be leveraged to build a holistic deception based defense. However, to the best of our knowledge, there has not been a work that provides a systematic retrospect of these three domains all together and investigates their integrated usage for orchestrated deceptions. Our paper aims to fill this gap.…
Intelligent autonomous agents will be widely present on the battlefield of the future. The proliferation of intelligent agents is the emerging reality of warfare, and they will form an ever growing fraction of total military assets. By necessity, intelligent autonomous cyber defense agents are likely to become primary cyber fighters on the future battlefield. Initial explorations have identified the key functions, components and their interactions for a potential reference architecture of such an agent. However, it is beyond the current state of AI to support an agent that could operate intell…
Attack--defense trees are used to describe security weaknesses of a system and possible countermeasures. In this paper, the connection between attack--defense trees and game theory is made explicit. We show that attack--defense trees and binary zero-sum two-player extensive form games have equivalent expressive power when considering satisfiability, in the sense that they can be converted into each other while preserving their outcome and their internal structure.
Networked-Control Systems (NCSs), a type of cyber-physical systems, consist of tightly integrated computing, communication and control technologies. While being very flexible environments, they are vulnerable to computing and networking attacks. Recent NCSs hacking incidents had major impact. They call for more research on cyber-physical security. Fears about the use of quantum computing to break current cryptosystems make matters worse. While the quantum threat motivated the creation of new disciplines to handle the issue, such as post-quantum cryptography, other fields have overlooked the ex…
