We propose a framework based on Recurrent Neural Networks (RNNs) to determine an optimal control strategy for a discrete-time system that is required to satisfy specifications given as Signal Temporal Logic (STL) formulae. RNNs can store information of a system over time, thus, enable us to determine satisfaction of the dynamic temporal requirements specified in STL formulae. Given a STL formula, a dataset of satisfying system executions and corresponding control policies, we can use RNNs to predict a control policy at each time based on the current and previous states of system. We use Control Barrier Functions (CBFs) to guarantee the safety of the predicted control policy. We validate our theoretical formulation and demonstrate its performance in an optimal control problem subject to partially unknown safety constraints through simulations.
This paper studies the controller synthesis problem for Linear Temporal Logic (LTL) specifications using (constrained) zonotope techniques. First, we implement (constrained) zonotope techniques to partition the state space and further to verify whether the LTL specification can be satisfied. Once the LTL specification can be satisfied, the next step is to design a controller to guarantee the satisfaction of the LTL specification for dynamic systems. Based on the verification of the LTL specification, an abstraction-based control design approach is proposed in this paper: a novel abstraction co…
Osaka University（大阪大学）；National Institute of Informatics（国立信息学研究所）
In this paper, we propose a control synthesis method for signal temporal logic (STL) specifications with neural networks (NNs). Most of the previous works consider training a controller for only a given STL specification. These approaches, however, require retraining the NN controller if a new specification arises and needs to be satisfied, which results in large consumption of memory and inefficient training. To tackle this problem, we propose to construct NN controllers by introducing encoder-decoder structured NNs with an attention mechanism. The encoder takes an STL formula as input and en…
Boston University（波士顿大学）；Commonwealth School（联邦学校）
We propose a policy search approach to learn controllers from specifications given as Signal Temporal Logic (STL) formulae. The system model, which is unknown but assumed to be an affine control system, is learned together with the control policy. The model is implemented as two feedforward neural networks (FNNs) - one for the drift, and one for the control directions. To capture the history dependency of STL specifications, we use a recurrent neural network (RNN) to implement the control policy. In contrast to prevalent model-free methods, the learning approach proposed here takes advantage o…
When designing large-scale distributed controllers, the information-sharing constraints between sub-controllers, as defined by a communication topology interconnecting them, are as important as the controller itself. Controllers implemented using dense topologies typically outperform those implemented using sparse topologies, but it is also desirable to minimize the cost of controller deployment. Motivated by the above, we introduce a compact but expressive graph recurrent neural network (GRNN) parameterization of distributed controllers that is well suited for distributed controller and commu…
We propose a counterexample-guided inductive synthesis framework for the formal synthesis of closed-form sampled-data controllers for nonlinear systems to meet STL specifications over finite-time trajectories. Rather than stating the STL specification for a single initial condition, we consider an (infinite and bounded) set of initial conditions. Candidate solutions are proposed using genetic programming, which evolves controllers based on a finite number of simulations. Subsequently, the best candidate is verified using reachability analysis; if the candidate solution does not satisfy the spe…
Specification-aware Robustness Margins for Symbolic Controllers
College of Computing, University Mohammed VI Polytechnic（Mohammed VI Polytechnic大学计算学院）；Université Paris-Saclay, CNRS, CentraleSupélec, Laboratoire des Signaux et Systèmes（巴黎萨克雷大学、法国国家科学研究中心、中央理工-高等电力学院、信号与系统实验室）
Resilient and Effort-Optimal Controller Synthesis under Temporal Logic Specifications
University Mohammed VI Polytechnic（穆罕默德六世 polytechnic 大学）；The Robert Bosch Centre for Cyber-Physical Systems, IISc（印度科学学院罗伯特博世网络物理系统中心）；Newcastle University（纽卡斯尔大学）；Max Planck Institute for Software Systems（马克斯·普朗克软件系统研究所）；University of Birmingham（伯明翰大学）
This paper introduces the safety controller architecture as a runtime assurance mechanism for system specifications expressed as safety properties in Linear Temporal Logic (LTL). The safety controller has three fundamental components: a performance controller, a backup controller, and an assurance mechanism. The assurance mechanism uses a monitor, constructed as a finite state machine (FSM), to analyze a suggested performance control input and search for system trajectories that are bad prefixes of the system specification. A fault flag from the assurance mechanism denotes a potentially danger…
In this paper, a method for learning a recurrent neural network (RNN) controller that maximizes the robustness of signal temporal logic (STL) specifications is presented. In contrast to previous methods, we consider synthesizing the RNN controller for which the user is able to select an STL specification arbitrarily from multiple STL specifications. To obtain such a controller, we propose a novel notion called STL2vec, which represents a vector representation of the STL specifications and exhibits their similarities. The construction of the STL2vec is useful since it allows us to enhance the e…
Air Force Research Laboratory（空军研究实验室）；University of Central Florida（中佛罗里达大学）；University of Colorado Boulder（科罗拉多大学博尔德分校）
Given a Markov decision process (MDP) and a linear-time ($ω$-regular or LTL) specification, the controller synthesis problem aims to compute the optimal policy that satisfies the specification. More recently, problems that reason over the asymptotic behavior of systems have been proposed through the lens of steady-state planning. This entails finding a control policy for an MDP such that the Markov chain induced by the solution policy satisfies a given set of constraints on its steady-state distribution. This paper studies a generalization of the controller synthesis problem for a linear-time …
This paper proposes an offline control algorithm, called Recurrent Model Predictive Control (RMPC), to solve large-scale nonlinear finite-horizon optimal control problems. It can be regarded as an explicit solver of traditional Model Predictive Control (MPC) algorithms, which can adaptively select appropriate model prediction horizon according to current computing resources, so as to improve the policy performance. Our algorithm employs a recurrent function to approximate the optimal policy, which maps the system states and reference values directly to the control inputs. The output of the lea…
University of Pennsylvania（宾夕法尼亚大学）；Rensselaer Polytechnic Institute（伦斯勒理工学院）
Motivated by the fragility of neural network (NN) controllers in safety-critical applications, we present a data-driven framework for verifying the risk of stochastic dynamical systems with NN controllers. Given a stochastic control system, an NN controller, and a specification equipped with a notion of trace robustness (e.g., constraint functions or signal temporal logic), we collect trajectories from the system that may or may not satisfy the specification. In particular, each of the trajectories produces a robustness value that indicates how well (severely) the specification is satisfied (v…
In this paper, we present a controller synthesis approach for wind turbine generators (WTG) and energy storage systems with metric temporal logic (MTL) specifications, with provable probabilistic guarantees in the stochastic environment of wind power generation. The MTL specifications are requirements for the grid frequency deviations, WTG rotor speed variations and the power flow constraints at different lines. We present the stochastic control bisimulation function, which bounds the divergence of the trajectories of a switched stochastic control system and the switched nominal control system…
There are spatio-temporal rules that dictate how robots should operate in complex environments, e.g., road rules govern how (self-driving) vehicles should behave on the road. However, seamlessly incorporating such rules into a robot control policy remains challenging especially for real-time applications. In this work, given a desired spatio-temporal specification expressed in the Signal Temporal Logic (STL) language, we propose a semi-supervised controller synthesis technique that is attuned to human-like behaviors while satisfying desired STL specifications. Offline, we synthesize a trajecto…
We consider abstraction-based design of output-feedback controllers for non-linear dynamical systems against specifications over state-based predicates in linear-time temporal logic (LTL). In this context, our contribution is two-fold: (I) we generalize feedback-refinement relations for abstraction-based output-feedback control to systems with arbitrary predicate and observation maps, and (II) we introduce a new algorithm for the synthesis of abstract output-feedback controllers w.r.t. LTL specifications over unobservable state-based predicates. Our abstraction-based output-feedback controller…
School for Engineering of Matter, Transport, and Energy, Arizona State University（亚利桑那州立大学物质、运输与能源工程学院）；Argonne National Laboratory（阿贡国家实验室）
In this paper, we present a provably correct controller synthesis approach for switched stochastic control systems with metric temporal logic (MTL) specifications with provable probabilistic guarantees. We first present the stochastic control bisimulation function for switched stochastic control systems, which bounds the trajectory divergence between the switched stochastic control system and its nominal deterministic control system in a probabilistic fashion. We then develop a method to compute optimal control inputs by solving an optimization problem for the nominal trajectory of the determi…
Oden Institute for Computational Engineering and Sciences, University of Texas, Austin（奥斯汀德克萨斯大学奥登计算工程与科学研究所）；University of Florida（佛罗里达大学）
Privacy is an important concern in various multiagent systems in which data collected from the agents are sensitive. We propose a differentially private controller synthesis approach for multi-agent systems subject to high-level specifications expressed in metric temporal logic (MTL). We consider a setting where each agent sends data to a cloud (computing station) through a set of local hubs and the cloud is responsible for computing the control inputs of the agents. Specifically, each agent adds privacy noise (e.g., Gaussian noise) point-wise in time to its own outputs before sharing them wit…
Counter-example guided Imitation Learning of Feedback Controllers from Temporal Logic Specifications
Univ. Grenoble Alpes（格勒诺布尔阿尔卑斯大学）；CNRS（法国国家科学研究中心）；Grenoble INP（格勒诺布尔理工学院）；VERIMAG（VERIMAG 实验室）；Decyphir SAS（Decyphir SAS 公司）；IIT Kanpur（印度理工学院坎普尔分校）；Aristotle University of Thessaloniki（塞萨洛尼基亚里士多德大学）
University of California, Berkeley（加州大学伯克利分校）；University of Michigan（密歇根大学）；Virginia Tech（弗吉尼亚理工大学）
Neural network controllers have become popular in control tasks thanks to their flexibility and expressivity. Stability is a crucial property for safety-critical dynamical systems, while stabilization of partially observed systems, in many cases, requires controllers to retain and process long-term memories of the past. We consider the important class of recurrent neural networks (RNN) as dynamic controllers for nonlinear uncertain partially-observed systems, and derive convex stability conditions based on integral quadratic constraints, S-lemma and sequential convexification. To ensure stabil…
In this paper, we investigate the controller design problem for linear disturbed systems under signal temporal logic (STL) specifications imposing both spatial and temporal constraints on system behavior. We first implement zonotope-based techniques to partition the state space into finite cells, then propose an evaluation mechanism to rearrange the time constraints of the STL specification, and finally decompose the global STL formula into finite local STL formulas. In this way, each cell has a local control design problem, which is further formulated into a local optimization problem. To dea…
We propose a symbolic self-triggered controller synthesis procedure for non-deterministic continuous-time nonlinear systems without stability assumptions. The goal is to compute a controller that satisfies two objectives. The first objective is represented as a specification in a fragment of LTL, which we call 2-LTL. The second one is an energy objective, in the sense that control inputs are issued only when necessary, which saves energy. To this end, we first quantise the state and input spaces, and then translate the controller synthesis problem to the computation of a winning strategy in a …
In this paper, we study the control of dynamical systems under temporal logic task specifications using gradient-based methods relying on quantitative measures that express the extent to which the tasks are satisfied. A class of controllers capable of providing satisfaction guarantees for simple systems and specifications is introduced and then extended for the case of unicycle-like dynamics. The possibility of combining such controllers in order to tackle more complex task specifications while retaining their computational efficiency is examined, and the practicalities related to an effective…
We extend previous work on symbolic self-triggered control for non-deterministic continuous-time nonlinear systems without stability assumptions to a larger class of specifications. Our goal is to synthesise a controller for two objectives: the first one is modelled as a right-recursive LTL formula, and the second one is to ensure that the average communication rate between the controller and the system stays below a given threshold. We translate the control problem to solving a mean-payoff parity game played on a discrete graph. Apart from extending the class of specifications, we propose a h…
Owing to their superior modeling capabilities, gated Recurrent Neural Networks, such as Gated Recurrent Units (GRUs) and Long Short-Term Memory networks (LSTMs), have become popular tools for learning dynamical systems. This paper aims to discuss how these networks can be adopted for the synthesis of Internal Model Control (IMC) architectures. To this end, first a gated recurrent network is used to learn a model of the unknown input-output stable plant. Then, a controller gated recurrent network is trained to approximate the model inverse. The stability of these networks, ensured by means of a…
We apply deep reinforcement learning (DRL) to design of a networked controller with network delays to complete a temporal control task that is described by a signal temporal logic (STL) formula. STL is useful to deal with a specification with a bounded time interval for a dynamical system. In general, an agent needs not only the current system state but also the past behavior of the system to determine a desired control action for satisfying the given STL formula. Additionally, we need to consider the effect of network delays for data transmissions. Thus, we propose an extended Markov decision…
This paper studies the synthesis of control policies for an agent that has to satisfy a temporal logic specification in a partially observable environment, in the presence of an adversary. The interaction of the agent (defender) with the adversary is modeled as a partially observable stochastic game. The goal is to generate a defender policy to maximize satisfaction of a given temporal logic specification under any adversary policy. The search for policies is limited to the space of finite state controllers, which leads to a tractable approach to determine policies. We relate the satisfaction …
This paper proposes a framework for automatic formal controller synthesis for general hybrid systems with a subset of safety and reachability specifications. The framework uses genetic programming to automatically co-synthesize controllers and candidate Lyapunov-like functions. These candidate Lyapunov-like functions are used to formally verify the control specification, and their correctness is proven using a Satisfiability Modulo Theories solver. The advantages of this approach are: no restriction is made to polynomial systems, the synthesized controllers are expressed as compact expressions…
This paper is concerned with a compositional approach for the construction of control barrier certificates for large-scale interconnected stochastic systems while synthesizing hybrid controllers against high-level logic properties. Our proposed methodology involves decomposition of interconnected systems into smaller subsystems and leverages the notion of control sub-barrier certificates of subsystems, enabling one to construct control barrier certificates of interconnected systems by employing some max-type small-gain conditions. The main goal is to synthesize hybrid controllers enforcing com…
This paper investigates the design of a robust fixed-order controller for single-input-single-output (SISO) polytopic systems with interval uncertainties, with the aim that the closed-loop stability is appropriately ensured and the performance specifications on sensitivity shaping are conformed in a specific finite frequency range. Utilizing the notion of generalized common Lyapunov strictly positive realness (CL-SPRness), the equivalence between strictly positive realness (SPRness) and strictly bounded realness (SBRness) is established; and then the specifications on robust stability and perf…
The paper presents a feedforward plus feedback controller structure with I and PI controllers for control of an integrating process with dead time. Guidelines for controller gain selection based on time domain specifications of damping factor and natural frequency are provided along with simulations indicating the selectivity of process response. The utility of proposed controller structure is shown by simulating the depth control of a nonlinear autonomous underwater vehicle system by the proposed controller structure.
Recurrent neural network-based robust control systems with regional properties and application to MPC design
Dipartimento di Elettronica, Informazione e Bioingegneria, Politecnico di Milano（电子、信息与生物工程系，米兰理工大学）；Istituto di Sistemi e Tecnologie Industriali Intelligenti per il Manifatturiero Avanzato, Consiglio Nazionale delle Ricerche（先进智能制造工业系统与技术研究所，国家研究院）
In this paper, we consider supervisory control of stochastic discrete event systems (SDESs) under linear temporal logic specifications. Applying the bounded synthesis, we reduce the supervisor synthesis into a problem of satisfying a safety condition. First, we consider a synthesis problem of a directed controller using the safety condition. We assign a negative reward to the unsafe states and introduce an expected return with a state-dependent discount factor. We compute a winning region and a directed controller with the maximum satisfaction probability using a dynamic programming method, wh…
Patching Control Lyapunov Barrier Functions for Temporal Logic Specifications with Bounded Controls
Massachusetts Institute of Technology（麻省理工学院）；University of Waterloo（滑铁卢大学）；MacLean Engineering & Marketing Co. Limited（麦克林工程与营销有限公司）；Hong Kong University of Science and Technology (Guangzhou)（香港科技大学（广州））
Urban Air Mobility (UAM), or the scenario where multiple manned and Unmanned Aerial Vehicles (UAVs) carry out various tasks over urban airspaces, is a transportation concept of the future that is gaining prominence. UAM missions with complex spatial, temporal and reactive requirements can be succinctly represented using Signal Temporal Logic (STL), a behavioral specification language. However, planning and control of systems with STL specifications is computationally intensive, usually resulting in planning approaches that do not guarantee dynamical feasibility, or control approaches that cann…
In this paper, we consider the problem of automatically designing a Rectified Linear Unit (ReLU) Neural Network (NN) architecture (number of layers and number of neurons per layer) with the assurance that it is sufficiently parametrized to control a nonlinear system; i.e. control the system to satisfy a given formal specification. This is unlike current techniques, which provide no assurances on the resultant architecture. Moreover, our approach requires only limited knowledge of the underlying nonlinear system and specification. We assume only that the specification can be satisfied by a Lips…
