Neural networks have shown great promises in planning, control, and general decision making for learning-enabled cyber-physical systems (LE-CPSs), especially in improving performance under complex scenarios. However, it is very challenging to formally analyze the behavior of neural network based planners for ensuring system safety, which significantly impedes their applications in safety-critical domains such as autonomous driving. In this work, we propose a hierarchical neural network based planner that analyzes the underlying physical scenarios of the system and learns a system-level behavior planning scheme with multiple scenario-specific motion-planning strategies. We then develop an efficient verification method that incorporates overapproximation of the system state reachable set and novel partition and union techniques for formally ensuring system safety under our physics-aware planner. With theoretical analysis, we show that considering the different physical scenarios and building a hierarchical planner based on such analysis may improve system safety and verifiability. We also empirically demonstrate the effectiveness of our approach and its advantage over other baselines in practical case studies of unprotected left turn and highway merging, two common challenging safety-critical tasks in autonomous driving.
High-speed trajectory planning through unknown environments requires algorithmic techniques that enable fast reaction times while maintaining safety as new information about the operating environment is obtained. The requirement of computational tractability typically leads to optimization problems that do not include the obstacle constraints (collision checks are done on the solutions) or use a convex decomposition of the free space and then impose an ad-hoc time allocation scheme for each interval of the trajectory. Moreover, safety guarantees are usually obtained by having a local planner t…
This study proposes an integrated task and motion planning method for dynamic locomotion in partially observable environments with multi-level safety guarantees. This layered planning framework is composed of a high-level symbolic task planner and a low-level phase-space motion planner. A belief abstraction at the task planning level enables belief estimation of dynamic obstacle locations and guarantees navigation safety with collision avoidance. The high-level task planner, i.e., a two-level navigation planner, employs linear temporal logic for a reactive game synthesis between the robot and …
Zhejiang University（浙江大学）；Huzhou Institute, Zhejiang University（浙江大学湖州研究院）
This paper proposes Elastic Tracker, a flexible trajectory planning framework that can deal with challenging tracking tasks with guaranteed safety and visibility. Firstly, an object detection and intension-free motion prediction method is designed. Then an occlusion-aware path finding method is proposed to provide a proper topology. A smart safe flight corridor generation strategy is designed with the guiding path. An analytical occlusion cost is evaluated. Finally, an effective trajectory optimization approach enables to generate a spatio-temporal optimal trajectory within the resultant fligh…
Follow-Bench: A Unified Motion Planning Benchmark for Socially-Aware Robot Person Following
Shenzhen Key Laboratory of Robotics and Computer Vision（深圳机器人与计算机视觉重点实验室）；Southern University of Science and Technology（南方科技大学）；Human-Robot Interfaces and Interaction Laboratory（人机交互实验室）；Istituto Italiano Di Tecnologia（意大利技术研究院）；Swiss Federal Technology Institute of Lausanne（洛桑联邦理工学院）
KIO-planner: Attention-Guided Single-Stage Motion Planning with Dual Mapping for UAV Navigation
Faculty of Applied Sciences（应用科学学院）；Macao Polytechnic University（澳门理工学院）；College of Animal Science and Technology（动物科学与技术学院）；Zhongkai University of Agriculture and Engineering（仲恺农业工程学院）；School of Economics and Management（经济管理学院）；South China Normal University（华南师范大学）；Information Engineering School（信息工程学院）
Mosaic: An Extensible Framework for Composing Rule-Based and Learned Motion Planners
Institute of Measurement and Control Systems, Karlsruhe Institute of Technology (KIT)（测量与控制系统的研究所，卡尔斯鲁厄理工学院）；FZI Research Center for Information Technology（信息技术研究所以）
Autonomous vehicles face tremendous challenges while interacting with human drivers in different kinds of scenarios. Developing control methods with safety guarantees while performing interactions with uncertainty is an ongoing research goal. In this paper, we present a real-time safe control framework using bi-level optimization with Control Barrier Function (CBF) that enables an autonomous ego vehicle to interact with human-driven cars in ramp merging scenarios with a consistent safety guarantee. In order to explicitly address motion uncertainty, we propose a novel extension of control barri…
This paper reports on developing an integrated framework for safety-aware informative motion planning suitable for legged robots. The information-gathering planner takes a dense stochastic map of the environment into account, while safety constraints are enforced via Control Barrier Functions (CBFs). The planner is based on the Incrementally-exploring Information Gathering (IIG) algorithm and allows closed-loop kinodynamic node expansion using a Model Predictive Control (MPC) formalism. Robotic exploration and information gathering problems are inherently path-dependent problems. That is, the …
Carnegie Mellon University（卡内基梅隆大学）；University of North Carolina at Charlotte（北卡罗来纳大学夏洛特分校）
With the increasing need for safe control in the domain of autonomous driving, model-based safety-critical control approaches are widely used, especially Control Barrier Function (CBF)-based approaches. Among them, Exponential CBF (eCBF) is particularly popular due to its realistic applicability to high-relative-degree systems. However, for most of the optimization-based controllers utilizing CBF-based constraints, solution feasibility is a common issue arising from potential conflict among different constraints. Moreover, how to incorporate uncertainty into the eCBF-based constraints in high-…
BOW: Bayesian Optimization over Windows for Motion Planning in Complex Environments
Department of Computer Science, University of New Orleans（新奥尔良大学计算机科学系）；School of Computing and Information Sciences, Florida International University（佛罗里达国际大学计算与信息科学学院）；Providence College, Department of Mathematics & Computer Science（普罗维登斯学院数学与计算机科学系）
For lateral vehicle dynamics, planning trajectories for lane-keeping and lane-change can be generalized as a path planning task to stabilize a vehicle onto a target lane, which is a fundamental element in nowadays autonomous driving systems. On the other hand, two-point steering for lane-change and lane-keeping has been investigated by researchers from psychology as a sensorimotor mechanism of human drivers. In the first part of this paper, using knowledge of geometric control, we will first design a path planner which satisfies five design objectives: generalization for different vehicle mode…
The goal of this paper is to develop a continuous optimization-based refinement of the reference trajectory to 'push it out' of the obstacle-occupied space in the global phase for Multi-rotor Aerial Vehicles in unknown environments. Our proposed approach comprises two planners: a global planner and a local planner. The global planner refines the initial reference trajectory when the trajectory goes either through an obstacle or near an obstacle and lets the local planner calculate a near-optimal control policy. The global planner comprises two convex programming approaches: the first one helps…
The ability to track a general walking path with specific timing is crucial to the operational safety and reliability of bipedal robots for avoiding dynamic obstacles, such as pedestrians, in complex environments. This paper introduces an online, full-body motion planner that generates the desired impact-aware motion for fully-actuated bipedal robotic walking. The main novelty of the proposed planner lies in its capability of producing desired motions in real-time that respect the discrete impact dynamics and the desired impact timing. To derive the proposed planner, a full-order hybrid dynami…
Sparse Planner: A Hybrid Planner for Efficient Sampling via a Conditional Variational Autoencoder
Technische Hochschule Ingolstadt (THI)（英戈尔施塔特应用技术大学）；German Aerospace Center (DLR)（德国航空航天中心）；German Research Center for Artificial Intelligence (DFKI)（德国人工智能研究中心）
Fast and safe navigation of dynamical systems through a priori unknown cluttered environments is vital to many applications of autonomous systems. However, trajectory planning for autonomous systems is computationally intensive, often requiring simplified dynamics that sacrifice safety and dynamic feasibility in order to plan efficiently. Conversely, safe trajectories can be computed using more sophisticated dynamic models, but this is typically too slow to be used for real-time planning. We propose a new algorithm FaSTrack: Fast and Safe Tracking for High Dimensional systems. A path or trajec…
Continuous-Time Gaussian Belief Trees for Motion Planning
Department of Aerospace Engineering Sciences at the University of Colorado Boulder（科罗拉多大学博尔德分校航空航天工程科学系）；Department of Aerospace and Ocean Engineering at Virginia Tech（弗吉尼亚理工学院航空航天与海洋工程系）
Stony Brook University（石溪大学）；University of Pennsylvania（宾夕法尼亚大学）
High-performance autonomy often must operate at the boundaries of safety. When external agents are present in a system, the process of ensuring safety without sacrificing performance becomes extremely difficult. In this paper, we present an approach to stress test such systems based on the rapidly exploring random tree (RRT) algorithm. We propose to find faults in such systems through adversarial agent perturbations, where the behaviors of other agents in an otherwise fixed scenario are modified. This creates a large search space of possibilities, which we explore both randomly and with a focu…
Carnegie Mellon University（卡内基梅隆大学）；Cleveland State University（克利夫兰州立大学）
Autonomous car racing is a challenging task, as it requires precise applications of control while the vehicle is operating at cornering speeds. Traditional autonomous pipelines require accurate pre-mapping, localization, and planning which make the task computationally expensive and environment-dependent. Recent works propose use of imitation and reinforcement learning to train end-to-end deep neural networks and have shown promising results for high-speed racing. However, the end-to-end models may be dangerous to be deployed on real systems, as the neural networks are treated as black-box mod…
Whole-body motion planning and safety-critical control for aerial manipulation
School of Mechanical and Aerospace Engineering, Nanyang Technological University, Singapore（机械与航空航天工程学院，南洋理工大学，新加坡）；Department of Aerospace Engineering, Seoul National University, South Korea（航空航天工程系，首尔国立大学，韩国）
Robot Local Planner: A Periodic Sampling-Based Motion Planner with Minimal Waypoints for Home Environments
Frontier Research Center, Toyota Motor Corporation（丰田汽车公司前沿研究中心）；Department of Information Science, Faculty of Information Science, Aichi Institute of Technology（爱知技术大学信息科学系）
The Chinese University of Hong Kong（香港中文大学）；Georgia Institute of Technology（佐治亚理工学院）；Southern University of Science and Technology（南方科技大学）；Shenzhen Research Institute of The Chinese University of Hong Kong（香港中文大学深圳研究院）
Balancing the trade-off between safety and efficiency is of significant importance for path planning under uncertainty. Many risk-aware path planners have been developed to explicitly limit the probability of collision to an acceptable bound in uncertain environments. However, convex obstacles or Gaussian uncertainties are usually assumed to make the problem tractable in the existing method. These assumptions limit the generalization and application of path planners in real-world implementations. In this article, we propose to apply deep learning methods to the sampling-based planner, developi…
Autonomous Systems Lab, ETH Zürich（苏黎世联邦理工学院自主系统实验室）
In order to enable Micro-Aerial Vehicles (MAVs) to assist in complex, unknown, unstructured environments, they must be able to navigate with guaranteed safety, even when faced with a cluttered environment they have no prior knowledge of. While trajectory optimization-based local planners have been shown to perform well in these cases, prior work either does not address how to deal with local minima in the optimization problem, or solves it by using an optimistic global planner. We present a conservative trajectory optimization-based local planner, coupled with a local exploration strategy that…
This study proposes a hierarchically integrated framework for safe task and motion planning (TAMP) of bipedal locomotion in a partially observable environment with dynamic obstacles and uneven terrain. The high-level task planner employs linear temporal logic (LTL) for a reactive game synthesis between the robot and its environment and provides a formal guarantee on navigation safety and task completion. To address environmental partial observability, a belief abstraction is employed at the high-level navigation planner to estimate the dynamic obstacles' location. Accordingly, a synthesized ac…
An ACO-MPC Framework for Energy-Efficient and Collision-Free Path Planning in Autonomous Maritime Navigation
School of Engineering, University of Strathclyde（斯特拉思克莱德大学工程学院）；James Watt School of Engineering, University of Glasgow（格拉斯哥大学詹姆斯·瓦特工程学院）；School of Power and energy, Northwestern Polytechnical University（西北工业大学动力与能源学院）；School of Engineering, University of Birmingham（伯明翰大学工程学院）
University of North Carolina Chapel Hill（北卡罗来纳大学教堂山分校）
In this paper, we perform safety and performance analysis of an autonomous vehicle that implements reactive planner and controller for navigating a race lap. Unlike traditional planning algorithms that have access to a map of the environment, reactive planner generates the plan purely based on the current input from sensors. Our reactive planner selects a waypoint on the local Voronoi diagram and we use a pure-pursuit controller to navigate towards the waypoint. Our safety and performance analysis has two parts. The first part demonstrates that the reactive planner computes a plan that is loca…
Control Barrier Functions (CBF) are widely used to enforce the safety-critical constraints on nonlinear systems. Recently, these functions are being incorporated into a path planning framework to design safety-critical path planners. However, these methods fall short of providing a realistic path considering both the algorithm's run-time complexity and enforcement of the safety-critical constraints. This paper proposes a novel motion planning approach using the well-known Rapidly Exploring Random Trees (RRT) algorithm that enforces both CBF and the robot Kinodynamic constraints to generate a s…
Action anticipation, intent prediction, and proactive behavior are all desirable characteristics for autonomous driving policies in interactive scenarios. Paramount, however, is ensuring safety on the road -- a key challenge in doing so is accounting for uncertainty in human driver actions without unduly impacting planner performance. This paper introduces a minimally-interventional safety controller operating within an autonomous vehicle control stack with the role of ensuring collision-free interaction with an externally controlled (e.g., human-driven) counterpart while respecting static obs…
Agile Decision-Making and Safety-Critical Motion Planning for Emergency Autonomous Vehicles
University of Hong Kong, Department of Mechanical Engineering（香港大学机械工程系）；National University of Singapore, Department of Civil and Environmental Engineering（新加坡国立大学土木与环境工程系）
Eindhoven University of Technology（埃因霍温理工大学）；Eindhoven AI Systems Institute（埃因霍温人工智能系统研究所）
Safe navigation around obstacles is a fundamental challenge for highly dynamic robots. The state-of-the-art approach for adapting simple reference path planners to complex robot dynamics using trajectory optimization and tracking control is brittle and requires significant replanning cycles. In this paper, we introduce a novel feedback motion planning framework that extends the applicability of low-order (e.g. position-/velocity-controlled) reference motion planners to high-order (e.g., acceleration-/jerk-controlled) robot models using motion prediction and reference governors. We use predicte…
Georgia Institute of Technology（佐治亚理工学院）；Carnegie Mellon University（卡内基梅隆大学）
This paper describes a hierarchical solution consisting of a multi-phase planner and a low-level safe controller to jointly solve the safe navigation problem in crowded, dynamic, and uncertain environments. The planner employs dynamic gap analysis and trajectory optimization to achieve collision avoidance with respect to the predicted trajectories of dynamic agents within the sensing and planning horizon and with robustness to agent uncertainty. To address uncertainty over the planning horizon and real-time safety, a fast reactive safe set algorithm (SSA) is adopted, which monitors and modifie…
Safe Planner: Empowering Safety Awareness in Large Pre-Trained Models for Robot Task Planning
Harbin Institute of Technology（哈尔滨工业大学）；Intelligent Science & Technology Academy Limited of CASIC（中国航天科工集团智能科技研究院有限公司）；Institute of Computer Application Technology, Norinco Group（中国兵器工业集团计算机应用技术研究所）
Connectivity technology has shown great potentials in improving the safety and efficiency of transportation systems by providing information beyond the perception and prediction capabilities of individual vehicles. However, it is expected that human-driven and autonomous vehicles, and connected and non-connected vehicles need to share the transportation network during the transition period to fully connected and automated transportation systems. Such mixed traffic scenarios significantly increase the complexity in analyzing system behavior and quantifying uncertainty for highly interactive sce…
Hong Kong Polytechnic University（香港理工大学）；The University of Hong Kong（香港大学）
For accomplishing a variety of missions in challenging environments, the capability of navigating with full autonomy while avoiding unexpected obstacles is the most crucial requirement for UAVs in real applications. In this paper, we proposed such a computationally efficient obstacle avoidance trajectory planner that can be used in cluttered unknown environments. Because of the narrow view field of single depth camera on a UAV, the information of obstacles around is quite limited thus the shortest entire path is difficult to achieve. Therefore we focus on the time cost of the trajectory planne…
