We propose a new specification language and control synthesis technique for single and multi-robot high-level tasks; these tasks include timing constraints and reaction to environmental events. Specifically, we define Event-based Signal Temporal Logic (STL) and use it to encode tasks that are reactive to uncontrolled environment events. Our control synthesis approach to Event-based STL tasks combines automata and control barrier functions to produce robot behaviors that satisfy the specification when possible. Our method automatically provides feedback to the user if an Event-based STL task can not be achieved. We demonstrate the effectiveness of the framework through simulations and physical demonstrations of multi-robot tasks.
Given a heterogeneous group of robots executing a complex task represented in Linear Temporal Logic, and a new set of tasks for the group, we define the task update problem and propose a framework for automatically updating individual robot tasks given their respective existing tasks and capabilities. Our heuristic, token-based, conflict resolution task allocation algorithm generates a near-optimal assignment for the new task. We demonstrate the scalability of our approach through simulations of multi-robot tasks.
This paper investigates the task coordination of multi-robot where each robot has a private individual temporal logic task specification; and also has to jointly satisfy a globally given collaborative temporal logic task specification. To efficiently generate feasible and optimized task execution plans for the robots, we propose a hierarchical multi-robot temporal task planning framework, in which a central server allocates the collaborative tasks to the robots, and then individual robots can independently synthesize their task execution plans in a decentralized manner. Furthermore, we propose…
This paper proposes a new highly scalable and asymptotically optimal control synthesis algorithm from linear temporal logic specifications, called $\text{STyLuS}^{*}$ for large-Scale optimal Temporal Logic Synthesis, that is designed to solve complex temporal planning problems in large-scale multi-robot systems. Existing planning approaches with temporal logic specifications rely on graph search techniques applied to a product automaton constructed among the robots. In our previous work, we have proposed a more tractable sampling-based algorithm that builds incrementally trees that approximate…
This paper proposes a new optimal control synthesis algorithm for multi-robot systems under global temporal logic tasks. Existing planning approaches under global temporal goals rely on graph search techniques applied to a product automaton constructed among the robots. In this paper, we propose a new sampling-based algorithm that builds incrementally trees that approximate the state-space and transitions of the synchronous product automaton. By approximating the product automaton by a tree rather than representing it explicitly, we require much fewer memory resources to store it and motion pl…
RWTH Aachen University（亚琛工业大学）；University of Genoa（热那亚大学）
In manufacturing, the increasing involvement of autonomous robots in production processes poses new challenges on the production management. In this paper we report on the usage of Optimization Modulo Theories (OMT) to solve certain multi-robot scheduling problems in this area. Whereas currently existing methods are heuristic, our approach guarantees optimality for the computed solution. We do not only present our final method but also its chronological development, and draw some general observations for the development of OMT-based approaches.
Uppsala University（乌普萨拉大学）；Washington University（华盛顿大学）；KTH Royal Institute of Technology（KTH皇家理工学院）
We develop an algorithm for the motion and task planning of a system comprised of multiple robots and unactuated objects under tasks expressed as Linear Temporal Logic (LTL) constraints. The robots and objects evolve subject to uncertain dynamics in an obstacle-cluttered environment. The key part of the proposed solution is the intelligent construction of a coupled transition system that encodes the motion and tasks of the robots and the objects. We achieve such a construction by designing appropriate adaptive control protocols in the lower level, which guarantee the safe robot navigation/obje…
We reformulate the signal temporal logic (STL) synthesis problem as a maximum a-posteriori (MAP) inference problem. To this end, we introduce the notion of random STL~(RSTL), which extends deterministic STL with random predicates. This new probabilistic extension naturally leads to a synthesis-as-inference approach. The proposed method allows for differentiable, gradient-based synthesis while extending the class of possible uncertain semantics. We demonstrate that the proposed framework scales well with GPU-acceleration, and present realistic applications of uncertain semantics in robotics tha…
Hanoi University of Industry（河内工业大学）；University of Economics - Technology for Industries（工业经济技术大学）
Mobile robots have been widely used in various aspects of human life. When a robot moves between different positions in the working area to perform the task, controlling motion to follow a pre-defined path is the primary task of a mobile robot. Furthermore, the robot must remain at its desired speed to cooperate with other agents. This paper presents a development of a motion controller, in which the fuzzy logic method is combined with a kinematic model of a differential drive robot. The simulation results are compared well with experimental results indicate that the method is effective and ap…
Singapore-MIT Alliance for Research and Technology（新加坡-麻省理工学院研究与技术联盟）；Boston University（波士顿大学）；Massachusetts Institute of Technology（麻省理工学院）
We consider the synthesis of control policies from temporal logic specifications for robots that interact with multiple dynamic environment agents. Each environment agent is modeled by a Markov chain whereas the robot is modeled by a finite transition system (in the deterministic case) or Markov decision process (in the stochastic case). Existing results in probabilistic verification are adapted to solve the synthesis problem. To partially address the state explosion issue, we propose an incremental approach where only a small subset of environment agents is incorporated in the synthesis proce…
Controlling contacts is truly challenging, and this has been a major hurdle to deploying industrial robots into unstructured/human-centric environments. More specifically, the main challenges are: (i) how to ensure stability at all times; (ii) how to satisfy task-specific performance specifications; (iii) how to achieve (i) and (ii) under environment uncertainty, robot parameters uncertainty, sensor and actuator time delays, external perturbations, etc. Here, we propose a new approach -- Convex Controller Synthesis (CCS) -- to tackle the above challenges based on robust control theory and conv…
In this paper, we propose a framework for the control of mobile robots subject to temporal logic specifications using barrier functions. Complex task specifications can be conveniently encoded using linear temporal logic. In particular, we consider a fragment of linear temporal logic which encompasses a large class of motion planning specifications for a robotic system. Control barrier functions have recently emerged as a convenient tool to guarantee reachability and safety for a system. In addition, they can be encoded as affine constraints in a quadratic program. In this paper, a fully autom…
We present a stepping stabilization control that addresses external push disturbances on bipedal walking robots. The stepping control is synthesized based on the step-to-step (S2S) dynamics of the robot that is controlled to have an approximately constant center of mass (COM) height. We first learn a linear S2S dynamics with bounded model discrepancy from the undisturbed walking behaviors of the robot, where the walking step size is taken as the control input to the S2S dynamics. External pushes are then considered as disturbances to the learned S2S (L-S2S) dynamics. We then apply the system-l…
We propose a predictive runtime monitoring framework that forecasts the distribution of future positions of mobile robots in order to detect and avoid impending property violations such as collisions with obstacles or other agents. Our approach uses a restricted class of temporal logic formulas to represent the likely intentions of the agents along with a combination of temporal logic-based optimal cost path planning and Bayesian inference to compute the probability of these intents given the current trajectory of the robot. First, we construct a large but finite hypothesis space of possible i…
Sapienza University（罗马第一大学）；Bosch Center for Artificial Intelligence（博世人工智能中心）
Signal Temporal Logic (STL) is an efficient technique for describing temporal constraints. It can play a significant role in robotic manipulation, for example, to optimize the robot performance according to task-dependent metrics. In this paper, we evaluate several STL robustness metrics of interest in robotic manipulation tasks and discuss a case study showing the advantages of using STL to define complex constraints. Such constraints can be understood as cost functions in task optimization. We show how STL-based cost functions can be optimized using a variety of off-the-shelf optimizers. We …
Boston University（波士顿大学）；University of Waterloo（滑铁卢大学）；Massachusetts Institute of Technology（麻省理工学院）
In this paper we present a method for automatically planning optimal paths for a group of robots that satisfy a common high level mission specification. Each robot's motion in the environment is modeled as a weighted transition system. The mission is given as a Linear Temporal Logic formula. In addition, an optimizing proposition must repeatedly be satisfied. The goal is to minimize the maximum time between satisfying instances of the optimizing proposition. Our method is guaranteed to compute an optimal set of robot paths. We utilize a timed automaton representation in order to capture the re…
SAP Labs, India（思爱普实验室（印度））；IIT Kanpur, India（印度坎普尔理工学院）
We address the path planning problem for a team of robots satisfying a complex high-level mission specification given in the form of an Linear Temporal Logic (LTL) formula. The state-of-the-art approach to this problem employs the automata-theoretic model checking technique to solve this problem. This approach involves computation of a product graph of the Buchi automaton generated from the LTL specification and a joint transition system which captures the collective motion of the robots and then computation of the shortest path using Dijkstra's shortest path algorithm. We propose MT*, an algo…
This paper aims for a controller that can stabilize a position-controlled humanoid robot when standing still or walking on synthetic grass even when subjected to external disturbances. Two types of controllers are designed and implemented: ankle strategy and stepping strategy. The robot's joints consist of position-controlled servos which can be complicated to model analytically due to nonlinearities and non-measurable parameters, hence the dynamic model of the humanoid robot is acquired using a non-recursive least squares system identification. This model is also used to design a Kalman Filte…
Boston University（波士顿大学）；Singapore-MIT Alliance for Research and Technology（新加坡-麻省理工学院研究与技术联盟）
In this paper, we present a method for optimal control synthesis of a plant that interacts with a set of agents in a graph-like environment. The control specification is given as a temporal logic statement about some properties that hold at the vertices of the environment. The plant is assumed to be deterministic, while the agents are probabilistic Markov models. The goal is to control the plant such that the probability of satisfying a syntactically co-safe Linear Temporal Logic formula is maximized. We propose a computationally efficient incremental approach based on the fact that temporal l…
Most autonomous robotic agents use logic inference to keep themselves to safe and permitted behaviour. Given a set of rules, it is important that the robot is able to establish the consistency between its rules, its perception-based beliefs, its planned actions and their consequences. This paper investigates how a robotic agent can use model checking to examine the consistency of its rules, beliefs and actions. A rule set is modelled by a Boolean evolution system with synchronous semantics, which can be translated into a labelled transition system (LTS). It is proven that stability and consist…
In this work, we propose an event-triggered con- trol framework for dynamical systems with temporal logical constraints. Event-triggered control methodologies have proven to be very efficient in reducing sensing, communication and computation costs. When a continuous feedback control is re- placed with an event-triggered strategy, the corresponding state trajectories also differ. In a system with logical constraints, such small deviation in the trajectory might lead to unsatisfiability of the logical constraints. In this work, we develop an approach where we ensure that the event-triggered sta…
In this paper, we develop a distributed intermittent communication and task planning framework for mobile robot teams. The goal of the robots is to accomplish complex tasks, captured by local Linear Temporal Logic formulas, and share the collected information with all other robots and possibly also with a user. Specifically, we consider situations where the robot communication capabilities are not sufficient to form reliable and connected networks while the robots move to accomplish their tasks. In this case, intermittent communication protocols are necessary that allow the robots to temporari…
Materials that respond to external stimuli by expanding or contracting provide a transduction route that integrates sensing and actuation powered directly by the stimuli. This motivates us to build colloidal scale robots using these materials that can morph into arbitrary configurations. For intelligent use of global stimuli in robotic systems, computation ability needs to be incorporated within them. The challenge is to design an architecture that is compact, material agnostic, stable under stochastic forces and can employ stimuli-responsive materials. We present an architecture that computes…
IST Austria (Institute of Science and Technology Austria)（奥地利科学技术研究所）；Indian Institute of Technology, Bombay（印度理工学院孟买分校）
We consider partially observable Markov decision processes (POMDPs), that are a standard framework for robotics applications to model uncertainties present in the real world, with temporal logic specifications. All temporal logic specifications in linear-time temporal logic (LTL) can be expressed as parity objectives. We study the qualitative analysis problem for POMDPs with parity objectives that asks whether there is a controller (policy) to ensure that the objective holds with probability 1 (almost-surely). While the qualitative analysis of POMDPs with parity objectives is undecidable, rece…
As robots gain capabilities to enter our human-centric world, they require formalism and algorithms that enable smart and efficient interactions. This is challenging, especially for robotic manipulators with complex tasks that may require collaboration with humans. Prior works approach this problem through reactive synthesis and generate strategies for the robot that guarantee task completion by assuming an adversarial human. While this assumption gives a sound solution, it leads to an "unfriendly" robot that is agnostic to the human intentions. We relax this assumption by formulating the prob…
Verified controller synthesis uses world models that comprise all potential behaviours of humans, robots, further equipment, and the controller to be synthesised. A world model enables quantitative risk assessment, for example, by stochastic model checking. Such a model describes a range of controller behaviours some of which -- when implemented correctly -- guarantee that the overall risk in the actual world is acceptable, provided that the stochastic assumptions have been made to the safe side. Synthesis then selects an acceptable-risk controller behaviour. However, because of crossing abstr…
University of Verona（维罗纳大学）；CMR Surgical Ltd（CMR外科有限公司）
Over the last decade, the use of robots in production and daily life has increased. With increasingly complex tasks and interaction in different environments including humans, robots are required a higher level of autonomy for efficient deliberation. Task planning is a key element of deliberation. It combines elementary operations into a structured plan to satisfy a prescribed goal, given specifications on the robot and the environment. In this manuscript, we present a survey on recent advances in the application of logic programming to the problem of task planning. Logic programming offers se…
For a nonlinear system (e.g. a robot) with its continuous state space trajectories constrained by a linear temporal logic specification, the synthesis of a low-level controller for mission execution often results in a non-convex optimization problem. We devise a new algorithm to solve this type of non-convex problems by formulating a rapidly-exploring random tree of barrier pairs, with each barrier pair composed of a quadratic barrier function and a full state feedback controller. The proposed method employs a rapid-exploring random tree to deal with the non-convex constraints and uses barrier…
In many applications, the integrals and derivatives of signals carry valuable information (e.g., cumulative success over a time window, the rate of change) regarding the behavior of the underlying system. In this paper, we extend the expressiveness of Signal Temporal Logic (STL) by introducing predicates that can define rich properties related to the integral and derivative of a signal. For control synthesis, the new predicates are encoded into mixed-integer linear inequalities and are used in the formulation of a mixed-integer linear program to find a trajectory that satisfies an STL specific…
In recent years, many works have investigated the feasibility of conversational robots for performing specific tasks, such as healthcare and interview. Along with this development comes a practical issue: how should we synthesize robotic voices to meet the needs of different situations? In this paper, we discuss this issue from three perspectives: 1) the difficulties of synthesizing non-verbal and interaction-oriented speech signals, particularly backchannels; 2) the scenario classification for robotic voice synthesis; 3) the ethical issues regarding the design of robot voice for its emotion a…
This paper presents an algorithmic framework for control synthesis of continuous dynamical systems subject to signal temporal logic (STL) specifications. We propose a novel algorithm to obtain a time-partitioned finite automaton from an STL specification, and introduce a multi-layered framework that utilizes this automaton to guide a sampling-based search tree both spatially and temporally. Our approach is able to synthesize a controller for nonlinear dynamics and polynomial predicate functions. We prove the correctness and probabilistic completeness of our algorithm, and illustrate the effici…
Many safety-critical systems must achieve high-level task specifications with guaranteed safety and correctness. Much recent progress towards this goal has been made through controller synthesis from temporal logic specifications. Existing approaches, however, have been limited to relatively short and simple specifications. Furthermore, existing methods either consider some prior discretization of the state-space, deal only with a convex fragment of temporal logic, or are not provably complete. We propose a scalable, provably complete algorithm that synthesizes continuous trajectories to satis…
University of Cambridge（剑桥大学）；CMR Surgical（CMR外科公司）
Most impedance control schemes in robotics implement a desired passive impedance, allowing for stable interaction between the controlled robot and the environment. However, there is little guidance on the selection of the desired impedance. In general, finding the best stiffness and damping parameters is a challenging task. This paper contributes to this problem by connecting impedance control to robust control, with the goal of shaping the robot performances via feedback. We provide a method based on linear matrix inequalities with sparsity constraints to derive impedance controllers that sat…
New Mexico State University（新墨西哥州立大学）；University of Notre Dame（圣母大学）
This paper considers an optimal task allocation problem for human robot collaboration in human robot systems with persistent tasks. Such human robot systems consist of human operators and intelligent robots collaborating with each other to accomplish complex tasks that cannot be done by either part alone. The system objective is to maximize the probability of successfully executing persistent tasks that are formulated as linear temporal logic specifications and minimize the average cost between consecutive visits of a particular proposition. This paper proposes to model the human robot collabo…
Motivated by the recent interest in cyber-physical and autonomous robotic systems, we study the problem of dynamically coupled multi-agent systems under a set of signal temporal logic tasks. In particular, the satisfaction of each of these signal temporal logic tasks depends on the behavior of a distinct set of agents. Instead of abstracting the agent dynamics and the temporal logic tasks into a discrete domain and solving the problem therein or using optimization-based methods, we derive collaborative feedback control laws. These control laws are based on a decentralized control barrier funct…
