The vulnerability of artificial intelligence (AI) and machine learning (ML) against adversarial disturbances and attacks significantly restricts their applicability in safety-critical systems including cyber-physical systems (CPS) equipped with neural network components at various stages of sensing and control. This paper addresses the reachable set estimation and safety verification problems for dynamical systems embedded with neural network components serving as feedback controllers. The closed-loop system can be abstracted in the form of a continuous-time sampled-data system under the control of a neural network controller. First, a novel reachable set computation method in adaptation to simulations generated out of neural networks is developed. The reachability analysis of a class of feedforward neural networks called multilayer perceptrons (MLP) with general activation functions is performed in the framework of interval arithmetic. Then, in combination with reachability methods developed for various dynamical system classes modeled by ordinary differential equations, a recursive algorithm is developed for over-approximating the reachable set of the closed-loop system. The safety verification for neural network control systems can be performed by examining the emptiness of the intersection between the over-approximation of reachable sets and unsafe sets. The effectiveness of the proposed approach has been validated with evaluations on a robotic arm model and an adaptive cruise control system.
Supervisory Measurement-Guided Noise Covariance Estimation: Discussing Forward and Reverse Differentiation
School of Data Science, the Chinese University of Hong Kong, Shenzhen（香港中文大学（深圳）数据科学学院）；School of Artificial Intelligence, the Chinese University of Hong Kong, Shenzhen（香港中文大学（深圳）人工智能学院）；School of Mechatronic Engineering and Automation, Shanghai University（上海大学机电工程与自动化学院）
