Neural networks have been increasingly applied for control in learning-enabled cyber-physical systems (LE-CPSs) and demonstrated great promises in improving system performance and efficiency, as well as reducing the need for complex physical models. However, the lack of safety guarantees for such neural network based controllers has significantly impeded their adoption in safety-critical CPSs. In this work, we propose a controller adaptation approach that automatically switches among multiple controllers, including neural network controllers, to guarantee system safety and improve energy efficiency. Our approach includes two key components based on formal methods and machine learning. First, we approximate each controller with a Bernstein-polynomial based hybrid system model under bounded disturbance, and compute a safe invariant set for each controller based on its corresponding hybrid system. Intuitively, the invariant set of a controller defines the state space where the system can always remain safe under its control. The union of the controllers' invariants sets then define a safe adaptation space that is larger than (or equal to) that of each controller. Second, we develop a deep reinforcement learning method to learn a controller switching strategy for reducing the control/actuation energy cost, while with the help of a safety guard rule, ensuring that the system stays within the safe space. Experiments on a linear adaptive cruise control system and a non-linear Van der Pol's oscillator demonstrate the effectiveness of our approach on energy saving and safety enhancement.
We propose a novel framework for modelling attack scenarios in cyber-physical control systems: we represent a cyber-physical system as a constrained switching system, where a single model embeds the dynamics of the physical process, the attack patterns, and the attack detection schemes. We show that this is compatible with established results in the analysis of hybrid automata, and, specifically, constrained switching systems. Moreover, we use the developed models to compute the impact of cyber attacks on the safety properties of the system. In particular, we characterise system safety as an a…
Cyber-physical systems (CPS) are required to operate safely under fault and malicious attacks. The simplex architecture and the recently proposed cyber resilient architectures, e.g., Byzantine fault tolerant++ (BFT++), provide safety for CPS under faults and malicious cyber attacks, respectively. However, these existing architectures make use of different timing parameters and implementations to provide safety, and are seemingly unrelated. In this paper, we propose an analytical framework to represent the simplex, BFT++ and other practical cyber resilient architectures (CRAs). We construct a h…
Tractable safety-ensuring algorithms for cyber-physical systems are important in critical applications. Approaches based on Control Barrier Functions assume continuous enforcement, which is not possible in an online fashion. This paper presents two tractable algorithms to ensure forward invariance of discrete-time controlled cyber-physical systems. Both approaches are based on Control Barrier Functions to provide strict mathematical safety guarantees. The first algorithm exploits Lipschitz continuity and formulates the safety condition as a robust program which is subsequently relaxed to a set…
The surging usage of electric vehicles (EVs) demand the robust deployment of trustworthy electric vehicle charging station (EVCS) with millisecond range latency and massive machine to machine communications where 5G could act. However, 5G suffers from inherent protocols, hardware, and software vulnerabilities that seriously threaten the communicating entities' cyber-physical security. To overcome these limitations in the EVCS system, this paper analyses the impact of False Data Injection (FDI) and Distributed Denial of Services (DDoS) attacks on the operation of EVCS. This work is an extension…
Cyber-physical systems (CPS) are required to satisfy safety constraints in various application domains such as robotics, industrial manufacturing systems, and power systems. Faults and cyber attacks have been shown to cause safety violations, which can damage the system and endanger human lives. Resilient architectures have been proposed to ensure safety of CPS under such faults and attacks via methodologies including redundancy and restarting from safe operating conditions. The existing resilient architectures for CPS utilize different mechanisms to guarantee safety, and currently there is no…
Safe and reliable electric vehicle charging stations (EVCSs) have become imperative in an intelligent transportation infrastructure. Over the years, there has been a rapid increase in the deployment of EVCSs to address the upsurging charging demands. However, advances in information and communication technologies (ICT) have rendered this cyber-physical system (CPS) vulnerable to suffering cyber threats, thereby destabilizing the charging ecosystem and even the entire electric grid infrastructure. This paper develops an advanced cybersecurity framework, where STRIDE threat modeling is used to i…
In this paper, we investigate joint sensor-actuator cyber attacks in discrete event systems. We assume that attackers can attack some sensors and actuators at the same time by altering observations and control commands. Because of the nondeterminism in observation and control caused by cyber attacks, the behavior of the supervised system becomes nondeterministic and may deviate from the safety specification. We define the upper-bound on all possible languages that can be generated by the supervised system to investigate the safety supervisory control problem under cyber attacks. After introduc…
High performance but unverified controllers, e.g., artificial intelligence-based (a.k.a. AI-based) controllers, are widely employed in cyber-physical systems (CPSs) to accomplish complex control missions. However, guaranteeing the safety and reliability of CPSs with this kind of controllers is currently very challenging, which is of vital importance in many real-life safety-critical applications. To cope with this difficulty, we propose in this work a Safe-visor architecture for sandboxing unverified controllers in CPSs operating in noisy environments (a.k.a. stochastic CPSs). The proposed arc…
Safety risk assessment is an essential process to ensure a dependable Cyber-Physical System (CPS) design. Traditional risk assessment considers only physical failures. For modern CPS, failures caused by cyber attacks are on the rise. The focus of latest research effort is on safety-security lifecycle integration and the expansion of modeling formalisms for risk assessment to incorporate security failures. The interaction between safety and security lifecycles and its impact on the overall system design, as well as the reliability loss resulting from ignoring security failures are some of the o…
The concept of traditional farming is changing rapidly with the introduction of smart technologies like the Internet of Things (IoT). Under the concept of smart agriculture, precision agriculture is gaining popularity to enable Decision Support System (DSS)-based farming management that utilizes widespread IoT sensors and wireless connectivity to enable automated detection and optimization of resources. Undoubtedly the success of the system would be impacted on crop productivity, where failure would impact severely. Like many other cyber-physical systems, one of the growing challenges to avoid…
Virginia Tech（弗吉尼亚理工大学）；University of Southern California（南加州大学）
Cyber-physical systems are often safety-critical in that violations of safety properties may lead to catastrophes. We propose a method to enforce the safety of systems with real-valued signals by synthesizing a runtime enforcer called the shield. Whenever the system violates a property, the shield, composed with the system, makes correction instantaneously to ensure that no erroneous output is generated by the combined system. While techniques for synthesizing Boolean shields are well understood, they do not handle real-valued signals ubiquitous in cyber-physical systems, meaning corrections m…
Vanderbilt University（范德堡大学）；University of Pennsylvania（宾夕法尼亚大学）
This paper presents the Neural Network Verification (NNV) software tool, a set-based verification framework for deep neural networks (DNNs) and learning-enabled cyber-physical systems (CPS). The crux of NNV is a collection of reachability algorithms that make use of a variety of set representations, such as polyhedra, star sets, zonotopes, and abstract-domain representations. NNV supports both exact (sound and complete) and over-approximate (sound) reachability algorithms for verifying safety and robustness properties of feed-forward neural networks (FFNNs) with various activation functions. F…
System complexity has become ubiquitous in the design, assessment, and implementation of practical and useful cyber-physical systems. This increased complexity is impacting the management of models necessary for designing cyber-physical systems that are able to take into account a number of ``-ilities'', such that they are safe and secure and ultimately resilient to disruption of service. We propose an ontological metamodel for system design that augments an already existing industry metamodel to capture the relationships between various model elements and safety, security, and resilient consi…
Current cyber-physical systems (CPS) are expected to accomplish complex tasks. To achieve this goal, high performance, but unverified controllers (e.g. deep neural network, black-box controllers from third parties) are applied, which makes it very challenging to keep the overall CPS safe. By sandboxing these controllers, we are not only able to use them but also to enforce safety properties over the controlled physical systems at the same time. However, current available solutions for sandboxing controllers are just applicable to deterministic (a.k.a. non-stochastic) systems, possibly affected…
Digitization and remote connectivity have enlarged the attack surface and made cyber systems more vulnerable. As attackers become increasingly sophisticated and resourceful, mere reliance on traditional cyber protection, such as intrusion detection, firewalls, and encryption, is insufficient to secure the cyber systems. Cyber resilience provides a new security paradigm that complements inadequate protection with resilience mechanisms. A Cyber-Resilient Mechanism (CRM) adapts to the known or zero-day threats and uncertainties in real-time and strategically responds to them to maintain critical …
Large steps are being taken by the industry and R&D organizations in automated driving technology development, as well as in setting up appropriate scenario-based safety assessment methods. In this paper a concept framework is proposed for a very specific group of vehicles: trucks in a platoon enabled by vehicle-to-vehicle (V2V) communication. The basis is formed by previously proposed scenario-based assessment methods for individual automated driving vehicles. These methods however do not consider the inter-vehicle communication component. It is shown how V2V communication interconnects vehic…
Complex dynamical systems rely on the correct deployment and operation of numerous components, with state-of-the-art methods relying on learning-enabled components in various stages of modeling, sensing, and control at both offline and online levels. This paper addresses the run-time safety monitoring problem of dynamical systems embedded with neural network components. A run-time safety state estimator in the form of an interval observer is developed to construct lower-bound and upper-bound of system state trajectories in run time. The developed run-time safety state estimator consists of two…
Generative AI Enabled Robust Sensor Placement in Cyber-Physical Power Systems: A Graph Diffusion Approach
Nanyang Technological University（南洋理工大学）；Suzhou Institute for Advanced Research, University of Science and Technology of China（中国科学技术大学苏州高等研究院）；Univ. Grenoble Alpes, CNRS, Grenoble INP（格勒诺布尔阿尔卑斯大学、法国国家科学研究中心、格勒诺布尔理工学院）；Sungkyunkwan University（成均馆大学）
Large-scale Load-Altering Attacks (LAAs) against Internet-of-Things (IoT) enabled high-wattage electrical appliances (e.g., wifi-enabled air-conditioners, electric vehicles, etc.) pose a serious threat to power systems' security and stability. In this work, a Cyber-Resilient Economic Dispatch (CRED) framework is presented to mitigate the destabilizing effect of LAAs while minimizing the overall operational cost by dynamically optimizing the frequency droop control gains of Inverter-Based Resources (IBRs). The system frequency dynamics incorporating both LAAs and the IBR droop control are model…
Reliability and fault tolerance are critical attributes of embedded cyber-physical systems that require a high safety-integrity level. For such systems, the use of formal functional safety specifications has been strongly advocated in most industrial safety standards, but reliability and fault tolerance have traditionally been treated as platform issues. We believe that addressing reliability and fault tolerance at the functional safety level widens the scope for resource optimization, targeting those functionalities that are safety-critical, rather than the entire platform. Moreover, for soft…
Collaborative Cyber-Physical Systems (CCPS) are systems that contain tightly coupled physical and cyber components, massively interconnected subsystems, and collaborate to achieve a common goal. The safety of a single Cyber-Physical System (CPS) can be achieved by following the safety standards such as ISO 26262 and IEC 61508 or by applying hazard analysis techniques. However, due to the complex, highly interconnected, heterogeneous, and collaborative nature of CCPS, a fault in one CPS's components can trigger many other faults in other collaborating CPSs. Therefore, a safety assurance techniq…
Although wireless technology is available for safety-critical applications, few applications have been used to improve train crossing safety. To prevent potential collisions between trains and vehicles, we present a Dedicated Short-Range Communication (DSRC)-enabled train safety communication system targeting to implement at unmanned crossings. Since our application's purpose is preventing collisions between trains and vehicles, we present a method to calculate the minimum required warning time for head-to-head collision at the train crossing. Furthermore, we define the best- and worst-case sc…
Stanford University（斯坦福大学）；NASA Ames Research Center（美国国家航空航天局埃姆斯研究中心）
Autonomous cyber-physical systems (CPS) can improve safety and efficiency for safety-critical applications, but require rigorous testing before deployment. The complexity of these systems often precludes the use of formal verification and real-world testing can be too dangerous during development. Therefore, simulation-based techniques have been developed that treat the system under test as a black box operating in a simulated environment. Safety validation tasks include finding disturbances in the environment that cause the system to fail (falsification), finding the most-likely failure, and …
While many companies worldwide are still striving to adjust to Industry 4.0 principles, the transition to Industry 5.0 is already underway. Under such a paradigm, Cyber-Physical Human-centered Systems (CPHSs) have emerged to leverage operator capabilities in order to meet the goals of complex manufacturing systems towards human-centricity, resilience and sustainability. This article first describes the essential concepts for the development of Industry 5.0 CPHSs and then analyzes the latest CPHSs, identifying their main design requirements and key implementation components. Moreover, the major…
Systems modeling practice lacks security analysis tools that can interface with modeling languages to facilitate security by design. Security by design is a necessity in the age of safety critical cyber-physical systems, where security violations can cause hazards. Currently, the overlap between security and safety is narrow. But deploying cyber-physical systems means that today's adversaries can intentionally trigger accidents. By implementing security assessment tools for modeling languages we are better able to address threats earlier in the system's lifecycle and, therefore, assure their s…
This article provides a tool for analyzing mechanisms that aim to achieve resilience against stealthy, or undetectable, attacks on cyber-physical systems (CPSs). We consider attackers who are able to corrupt all of the inputs and outputs of the system. To counter such attackers, a response scheme must be implemented that keeps the attacker from corrupting the inputs and outputs of the system for certain periods of time. To aid in the design of such a response scheme, our tool provides sufficient lengths for these periods of time in order to ensure safety with a particular probability. We provi…
Southern University of Science and Technology（南方科技大学）；Research Institute for Trustworthy Autonomous Systems（可信自主系统研究院）
Providing safety guarantees for Autonomous Vehicle (AV) systems with machine-learning-based controllers remains a challenging issue. In this work, we propose Simplex-Drive, a framework that can achieve runtime safety assurance for machine-learning enabled controllers of AVs. The proposed Simplex-Drive consists of an unverified Deep Reinforcement Learning (DRL)-based advanced controller (AC) that achieves desirable performance in complex scenarios, a Velocity-Obstacle (VO) based baseline safe controller (BC) with provably safety guarantees, and a verified mode management unit that monitors the …
The design of attacks for cyber physical systems is critical to assess CPS resilience at design time and run-time, and to generate rich datasets from testbeds for research. Attacks against cyber physical systems distinguish themselves from IT attacks in that the main objective is to harm the physical system. Therefore, both cyber and physical system knowledge are needed to design such attacks. The current practice to generate attacks either focuses on the cyber part of the system using IT cyber security existing body of knowledge, or uses heuristics to inject attacks that could potentially har…
Connected and Automated Vehicle (CAV) technology has the potential to greatly improve transportation mobility, safety, and energy efficiency. However, ubiquitous vehicular connectivity also opens up the door for cyber-attacks. In this study, we investigate cybersecurity risks of a representative cooperative traffic management application, i.e., highway on-ramp merging, in a mixed traffic environment. We develop threat models with two trajectory spoofing strategies on CAVs to create traffic congestion, and we also devise an attack-resilient strategy for system defense. Furthermore, we leverage …
Cyber-Physical Security and Safety of Autonomous Connected Vehicles: Optimal Control Meets Multi-Armed Bandit Learning
Centre for Wireless Communications (CWC), University of Oulu, Finland（奥卢大学无线通信中心（CWC））；WINLAB, Dept. of ECE, Rutgers University, New Brunswick, NJ, USA（罗格斯大学WINLAB，电子与计算机工程系，新泽西州新布朗斯维尔，美国）
Cyber-physical systems (CPS) greatly benefit by using machine learning components that can handle the uncertainty and variability of the real-world. Typical components such as deep neural networks, however, introduce new types of hazards that may impact system safety. The system behavior depends on data that are available only during runtime and may be different than the data used for training. Out-of-distribution data may lead to a large error and compromise safety. The paper considers the problem of efficiently detecting out-of-distribution data in CPS control systems. Detection must be robu…
Engineering safe and secure cyber-physical systems requires system engineers to develop and maintain a number of model views, both dynamic and static, which can be seen as algebras. We posit that verifying the composition of requirement, behavioral, and architectural models using category theory gives rise to a strictly compositional interpretation of cyber-physical systems theory, which can assist in the modeling and analysis of safety-critical cyber-physical systems.
Correct-by-construction synthesis is a cornerstone of the confluence of formal methods and control theory towards designing safety-critical systems. Instead of following the time-tested, albeit laborious (re)design-verify-validate loop, correct-by-construction methodology advocates the use of continual refinements of formal requirements -- connected by chains of formal proofs -- to build a system that assures the correctness by design. A remarkable progress has been made in scaling the scope of applicability of correct-by-construction synthesis -- with a focus on cyber-physical systems that ti…
