In this paper, we develop safe reinforcement-learning-based controllers for systems tasked with accomplishing complex missions that can be expressed as linear temporal logic specifications, similar to those required by search-and-rescue missions. We decompose the original mission into a sequence of tracking sub-problems under safety constraints. We impose the safety conditions by utilizing barrier functions to map the constrained optimal tracking problem in the physical space to an unconstrained one in the transformed space. Furthermore, we develop policies that intermittently update the control signal to solve the tracking sub-problems with reduced burden in the communication and computation resources. Subsequently, an actor-critic algorithm is utilized to solve the underlying Hamilton-Jacobi-Bellman equations. Finally, we support our proposed framework with stability proofs and showcase its efficacy via simulation results.
Control schemes for autonomous systems are often designed in a way that anticipates the worst case in any situation. At runtime, however, there could exist opportunities to leverage the characteristics of specific environment and operation context for more efficient control. In this work, we develop an online intermittent-control framework that combines formal verification with model-based optimization and deep reinforcement learning to opportunistically skip certain control computation and actuation to save actuation energy and computational resources without compromising system safety. Exper…
In systems where the ability to actuate is a scarce resource, e.g., spacecrafts, it is desirable to only apply a given controller in an intermittent manner--with periods where the controller is on and periods where it is off. Motivated by the event-triggered control paradigm, where state-dependent triggers are utilized in a sample-and-hold context, we generalize this concept to include state triggers where the controller is off thereby creating a framework for intermittent control. Our approach utilizes certificates--either Lyapunov or barrier functions--to design intermittent trigger laws tha…
Oden Institute for Computational Engineering and Sciences, University of Texas, Austin（得克萨斯大学奥斯汀分校奥登计算工程与科学研究所）；University of Florida（佛罗里达大学）
This paper develops a controller synthesis approach for a multi-agent system (MAS) with intermittent communication. We adopt a leader-follower scheme, where a mobile leader with absolute position sensors switches among a set of followers without absolute position sensors to provide each follower with intermittent state information.We model the MAS as a switched system. The followers are to asymptotically reach a predetermined consensus state. To guarantee the stability of the switched system and the consensus of the followers, we derive maximum and minimal dwell-time conditions to constrain th…
University of Pennsylvania（宾夕法尼亚大学）；University of Virginia（弗吉尼亚大学）；Stanford University（斯坦福大学）；Washington University of St. Louis（圣路易斯华盛顿大学）
This paper addresses a safe planning and control problem for mobile robots operating in communication- and sensor-limited dynamic environments. In this case the robots cannot sense the objects around them and must instead rely on intermittent, external information about the environment, as e.g., in underwater applications. The challenge in this case is that the robots must plan using only this stale data, while accounting for any noise in the data or uncertainty in the environment. To address this challenge we propose a compositional technique which leverages neural networks to quickly plan an…
The unknown sharp changes of vehicle acceleration rates, also called the unknown jerk dynamics, may significantly affect the driving performance of the leader vehicle in a platoon, resulting in more drastic car-following movements in platooning tracking control, which could cause safety and traffic capacity concerns. To address these issues, in this paper, we investigate cooperative platooning tracking control and intermittent optimization problems for connected automated vehicles (CAVs) with a nonlinear car-following model. We assume that the external inputs of the leader CAV contain unknown …
This paper introduces the safety controller architecture as a runtime assurance mechanism for system specifications expressed as safety properties in Linear Temporal Logic (LTL). The safety controller has three fundamental components: a performance controller, a backup controller, and an assurance mechanism. The assurance mechanism uses a monitor, constructed as a finite state machine (FSM), to analyze a suggested performance control input and search for system trajectories that are bad prefixes of the system specification. A fault flag from the assurance mechanism denotes a potentially danger…
The challenges in providing convincing arguments for safe and correct behavior of automated driving (AD) systems have so far hindered their widespread commercial deployment. Conventional development approaches such as testing and simulation are limited by non-exhaustive analysis, and can thus not guarantee correctness in all possible scenarios. Formal methods is an approach to provide mathematical proofs of correctness, using a model of a system, that could be used to give the necessary arguments. This paper investigates the use of differential dynamic logic and the deductive verification tool…
We consider the pressing question of how to model, verify, and ensure that autonomous systems meet certain \textit{obligations} (like the obligation to respect traffic laws), and refrain from impermissible behavior (like recklessly changing lanes). Temporal logics are heavily used in autonomous system design; however, as we illustrate here, temporal (alethic) logics alone are inappropriate for reasoning about obligations of autonomous systems. This paper proposes the use of Dominance Act Utilitarianism (DAU), a deontic logic of agency, to encode and reason about obligations of autonomous syste…
We propose a framework based on Recurrent Neural Networks (RNNs) to determine an optimal control strategy for a discrete-time system that is required to satisfy specifications given as Signal Temporal Logic (STL) formulae. RNNs can store information of a system over time, thus, enable us to determine satisfaction of the dynamic temporal requirements specified in STL formulae. Given a STL formula, a dataset of satisfying system executions and corresponding control policies, we can use RNNs to predict a control policy at each time based on the current and previous states of system. We use Contro…
In Urban Air Mobility (UAM) networks, takeoff and landing sites, called vertiports, are likely to experience intermittent closures due to, e.g., adverse weather. To ensure safety, all in-flight Urban Air Vehicles (UAVs) in a UAM network must therefore have alternative landing sites with sufficient landing capacity in the event of a vertiport closure. In this paper, we study the problem of safety verification of UAM schedules in the face of vertiport closures. We first provide necessary and sufficient conditions for a given UAM schedule to be safe in the sense that, if a vertiport closure occur…
Cyber-physical systems are often safety-critical and their correctness is crucial, as in the case of automated driving. Using formal mathematical methods is one way to guarantee correctness. Though these methods have shown their usefulness, care must be taken as modeling errors might result in proving a faulty controller safe, which is potentially catastrophic in practice. This paper deals with two such modeling errors in differential dynamic logic. Differential dynamic logic is a formal specification and verification language for hybrid systems, which are mathematical models of cyber-physical…
Boston University（波士顿大学）；Commonwealth School（联邦学校）
We propose a policy search approach to learn controllers from specifications given as Signal Temporal Logic (STL) formulae. The system model, which is unknown but assumed to be an affine control system, is learned together with the control policy. The model is implemented as two feedforward neural networks (FNNs) - one for the drift, and one for the control directions. To capture the history dependency of STL specifications, we use a recurrent neural network (RNN) to implement the control policy. In contrast to prevalent model-free methods, the learning approach proposed here takes advantage o…
This paper presents the concept of an adaptive safe padding that forces Reinforcement Learning (RL) to synthesise optimal control policies while ensuring safety during the learning process. Policies are synthesised to satisfy a goal, expressed as a temporal logic formula, with maximal probability. Enforcing the RL agent to stay safe during learning might limit the exploration, however we show that the proposed architecture is able to automatically handle the trade-off between efficient progress in exploration (towards goal satisfaction) and ensuring safety. Theoretical guarantees are available…
Smart cities are revolutionizing the transportation infrastructure by the integration of technology. However, ensuring that various transportation system components are operating as expected and in a safe manner is a great challenge. In this work, we propose the use of formal methods as a means to specify and reason about the traffic network's complex properties. Formal methods provide a flexible tool to define the safe operation of the traffic network by capturing non-conforming behavior, exploring various possible states of the traffic scene, and detecting any inconsistencies within it. Henc…
We present SaSTL -- a novel Spatial Aggregation Signal Temporal Logic -- for the efficient runtime monitoring of safety and performance requirements in smart cities. We first describe a study of over 1,000 smart city requirements, some of which can not be specified using existing logic such as Signal Temporal Logic (STL) and its variants. To tackle this limitation, we develop two new logical operators in SaSTL to augment STL for expressing spatial aggregation and spatial counting characteristics that are commonly found in real city requirements. We also develop efficient monitoring algorithms …
A fail-operational system for highly automated driving must complete the driving task even in the presence of a failure. This requires redundant architectures and a mechanism to reconfigure the system in case of a failure. Therefore, an arbitration logic is used. For functional safety, the switch-over to a fall-back level must be conducted in the presence of any electric and electronic failure. To provide evidence for a safety argumentation in compliance with ISO 26262, verification of the arbitration logic is necessary. The verification process provides confirmation of the correct failure rea…
Many safety-critical systems must achieve high-level task specifications with guaranteed safety and correctness. Much recent progress towards this goal has been made through controller synthesis from temporal logic specifications. Existing approaches, however, have been limited to relatively short and simple specifications. Furthermore, existing methods either consider some prior discretization of the state-space, deal only with a convex fragment of temporal logic, or are not provably complete. We propose a scalable, provably complete algorithm that synthesizes continuous trajectories to satis…
University of Southern California（南加州大学）；Toyota Motor North America R&D（丰田汽车北美研发中心）
In this paper, we consider the problem of synthesizing a controller in the presence of uncertainty such that the resulting closed-loop system satisfies certain hard constraints while optimizing certain (soft) performance objectives. We assume that the hard constraints encoding safety or mission-critical task objectives are expressed using Signal Temporal Logic (STL), while performance is quantified using standard cost functions on system trajectories. In order to prioritize the satisfaction of the hard STL constraints, we utilize the framework of control barrier functions (CBFs) and algorithmi…
LogicGuard: Improving Embodied LLM agents through Temporal Logic based Critics
Department of Mechanical Engineering, University of California at Santa Barbara（加州大学圣芭芭拉分校机械工程系）；Department of Electrical and Computer Engineering, Michigan State University（密歇根州立大学电气与计算机工程系）
