Multi-task Safe Reinforcement Learning for Navigating Intersections in Dense Traffic
Institute of Automation, Chinese Academy of Sciences（中国科学院自动化研究所）；College of Artificial Intelligence, University of Chinese Academy of Sciences（中国科学院大学人工智能学院）；School of Automation and Electrical Engineering, University of Science and Technology Beijing（北京科技大学自动化与电气工程学院）
Multi-task intersection navigation including the unprotected turning left, turning right, and going straight in dense traffic is still a challenging task for autonomous driving. For the human driver, the negotiation skill with other interactive vehicles is the key to guarantee safety and efficiency. However, it is hard to balance the safety and efficiency of the autonomous vehicle for multi-task intersection navigation. In this paper, we formulate a multi-task safe reinforcement learning with social attention to improve the safety and efficiency when interacting with other traffic participants. Specifically, the social attention module is used to focus on the states of negotiation vehicles. In addition, a safety layer is added to the multi-task reinforcement learning framework to guarantee safe negotiation. We compare the experiments in the simulator SUMO with abundant traffic flows and CARLA with high-fidelity vehicle models, which both show that the proposed algorithm can improve safety with consistent traffic efficiency for multi-task intersection navigation.
The University of Pennsylvania（宾夕法尼亚大学）；The University of Virginia（弗吉尼亚大学）；Honda Research Institute（本田研究所）；The Georgia Institute of Technology（佐治亚理工学院）
Providing an efficient strategy to navigate safely through unsignaled intersections is a difficult task that requires determining the intent of other drivers. We explore the effectiveness of Deep Reinforcement Learning to handle intersection problems. Using recent advances in Deep RL, we are able to learn policies that surpass the performance of a commonly-used heuristic approach in several metrics including task completion time and goal success rate and have limited ability to generalize. We then explore a system's ability to learn active sensing behaviors to enable navigating safely in the c…
Stanford University（斯坦福大学）；Honda Research Institute（本田研究院）
Navigating urban environments represents a complex task for automated vehicles. They must reach their goal safely and efficiently while considering a multitude of traffic participants. We propose a modular decision making algorithm to autonomously navigate intersections, addressing challenges of existing rule-based and reinforcement learning (RL) approaches. We first present a safe RL algorithm relying on a model-checker to ensure safety guarantees. To make the decision strategy robust to perception errors and occlusions, we introduce a belief update technique using a learning based approach. …
Navigating the Proximity-Safety Balance: Constraint Decomposition for Human Following in Pedestrian Crowds
University of Pennsylvania（宾夕法尼亚大学）；University of California, Riverside（加州大学河滨分校）；Stanford University（斯坦福大学）；NVIDIA Research（英伟达研究院）；Georgia Institute of Technology（佐治亚理工学院）
We propose a safe DRL approach for autonomous vehicle (AV) navigation through crowds of pedestrians while making a left turn at an unsignalized intersection. Our method uses two long-short term memory (LSTM) models that are trained to generate the perceived state of the environment and the future trajectories of pedestrians given noisy observations of their movement. A future collision prediction algorithm based on the future trajectories of the ego vehicle and pedestrians is used to mask unsafe actions if the system predicts a collision. The performance of our approach is evaluated in two exp…
University of Illinois at Urbana-Champaign（伊利诺伊大学厄巴纳-香槟分校）
Navigation through uncontrolled intersections is one of the key challenges for autonomous vehicles. Identifying the subtle differences in hidden traits of other drivers can bring significant benefits when navigating in such environments. We propose an unsupervised method for inferring driver traits such as driving styles from observed vehicle trajectories. We use a variational autoencoder with recurrent neural networks to learn a latent representation of traits without any ground truth trait labels. Then, we use this trait representation to learn a policy for an autonomous vehicle to navigate …
University of North Carolina Chapel Hill（北卡罗来纳大学教堂山分校）
In this paper, we perform safety and performance analysis of an autonomous vehicle that implements reactive planner and controller for navigating a race lap. Unlike traditional planning algorithms that have access to a map of the environment, reactive planner generates the plan purely based on the current input from sensors. Our reactive planner selects a waypoint on the local Voronoi diagram and we use a pure-pursuit controller to navigate towards the waypoint. Our safety and performance analysis has two parts. The first part demonstrates that the reactive planner computes a plan that is loca…
University of Parma（帕尔马大学）；University of Bologna（博洛尼亚大学）；VisLab srl, an Ambarella Inc. company（VisLab 公司，安霸公司旗下企业）
Navigating through intersections is one of the main challenging tasks for an autonomous vehicle. However, for the majority of intersections regulated by traffic lights, the problem could be solved by a simple rule-based method in which the autonomous vehicle behavior is closely related to the traffic light states. In this work, we focus on the implementation of a system able to navigate through intersections where only traffic signs are provided. We propose a multi-agent system using a continuous, model-free Deep Reinforcement Learning algorithm used to train a neural network for predicting bo…
Risk-Aware Reinforcement Learning for Autonomous Driving: Improving Safety When Driving through Intersection
School of Automotive Studies, Tongji University（同济大学汽车学院）；Department of Aeronautical and Aviation Engineering, The Hong Kong Polytechnic University（香港理工大学航空及航空工程系）
SinD 2.0: A Multi-City UAV Dataset with Semantic Risk Annotations for SOTIF-Oriented Safety Validation at Signalized Intersections
Tsinghua University（清华大学）；Beijing Institute of Technology（北京理工大学）；Guangzhou Automobile Group Co., Ltd.（广州汽车集团股份有限公司）；Jilin University（吉林大学）；Chang’an University（长安大学）；Chongqing University（重庆大学）；Southwest University（西南大学）；Wuhan University of Technology（武汉理工大学）；Virginia Tech Transportation Institute（弗吉尼亚理工大学交通研究所）；Virginia Tech（弗吉尼亚理工大学）
Stanford University（斯坦福大学）；Honda Research Institute（本田研究所）
Urban intersections represent a complex environment for autonomous vehicles with many sources of uncertainty. The vehicle must plan in a stochastic environment with potentially rapid changes in driver behavior. Providing an efficient strategy to navigate through urban intersections is a difficult task. This paper frames the problem of navigating unsignalized intersections as a partially observable Markov decision process (POMDP) and solves it using a Monte Carlo sampling method. Empirical results in simulation show that the resulting policy outperforms a threshold-based heuristic strategy on s…
Intersections are hazardous places. Threats arise from interactions among pedestrians, bicycles and vehicles, more complicated vehicle trajectories in the absence of lane markings, phases that prevent determining who has the right of way, invisible vehicle approaches, vehicle obstructions, and illegal movements. These challenges are not fully addressed by the "road diet" and road redesign prescribed in Vision Zero plans, nor will they be completely overcome by autonomous vehicles with their many sensors and tireless attention to surroundings. Accidents can also occur because drivers, cyclists …
Assessing collision risk is a critical challenge to effective traffic safety management. The deployment of unmanned aerial vehicles (UAVs) to address this issue has shown much promise, given their wide visual field and movement flexibility. This research demonstrates the application of UAVs and V2X connectivity to track the movement of road users and assess potential collisions at intersections. The study uses videos captured by UAVs. The proposed method combines deep-learning based tracking algorithms and time-to-collision tasks. The results not only provide beneficial information for vehicle…
For mobile robots navigating on sidewalks, it is essential to be able to safely cross street intersections. Most existing approaches rely on the recognition of the traffic light signal to make an informed crossing decision. Although these approaches have been crucial enablers for urban navigation, the capabilities of robots employing such approaches are still limited to navigating only on streets containing signalized intersections. In this paper, we address this challenge and propose a multimodal convolutional neural network framework to predict the safety of a street intersection for crossin…
Traffic signal control is safety-critical for our daily life. Roughly one-quarter of road accidents in the U.S. happen at intersections due to problematic signal timing, urging the development of safety-oriented intersection control. However, existing studies on adaptive traffic signal control using reinforcement learning technologies have focused mainly on minimizing traffic delay but neglecting the potential exposure to unsafe conditions. We, for the first time, incorporate road safety standards as enforcement to ensure the safety of existing reinforcement learning methods, aiming toward ope…
Autonomous Vehicles (AVs) i.e., self-driving cars, operate in a safety critical domain, since errors in the autonomous driving software can lead to huge losses. Statistically, road intersections which are a part of the AVs operational design domain (ODD), have some of the highest accident rates. Hence, testing AVs to the limits on road intersections and assuring their safety on road intersections is pertinent, and thus the focus of this paper. We present a situation coverage-based (SitCov) AV-testing framework for the verification and validation (V&V) and safety assurance of AVs, developed in …
Recent studies show that 85% of women have changed their traveled route to avoid harassment and assault. Despite this, current mapping tools do not empower users with information to take charge of their personal safety. We propose SafeRoute, a novel solution to the problem of navigating cities and avoiding street harassment and crime. Unlike other street navigation applications, SafeRoute introduces a new type of path generation via deep reinforcement learning. This enables us to successfully optimize for multi-criteria path-finding and incorporate representation learning within our framework.…
Autonomous driving at intersections is one of the most complicated and accident-prone traffic scenarios, especially with mixed traffic participants such as vehicles, bicycles and pedestrians. The driving policy should make safe decisions to handle the dynamic traffic conditions and meet the requirements of on-board computation. However, most of the current researches focuses on simplified intersections considering only the surrounding vehicles and idealized traffic lights. This paper improves the integrated decision and control framework and develops a learning-based algorithm to deal with com…
In classical reinforcement learning, when exploring an environment, agents accept arbitrary short term loss for long term gain. This is infeasible for safety critical applications, such as robotics, where even a single unsafe action may cause system failure. In this paper, we address the problem of safely exploring finite Markov decision processes (MDP). We define safety in terms of an, a priori unknown, safety constraint that depends on states and actions. We aim to explore the MDP under this constraint, assuming that the unknown function satisfies regularity conditions expressed via a Gaussi…
Intersection of Reinforcement Learning and Bayesian Optimization for Intelligent Control of Industrial Processes: A Safe MPC-based DPG using Multi-Objective BO
Department of Mechanical Engineering, Clemson University（机械工程系，克莱姆森大学）
University of Waterloo（滑铁卢大学）；Waytous Inc.（Waytous公司）；Autonomous Driving Lab, Tencent（腾讯自动驾驶实验室）；Tsinghua National Laboratory for Information Science and Technology, Department of Automation, Tsinghua University（清华大学自动化系清华大学信息科学与技术国家实验室）
Left-turn planning is one of the formidable challenges for autonomous vehicles, especially at unsignalized intersections due to the unknown intentions of oncoming vehicles. This paper addresses the challenge by proposing a critical turning point (CTP) based hierarchical planning approach. This includes a high-level candidate path generator and a low-level partially observable Markov decision process (POMDP) based planner. The proposed (CTP) concept, inspired by human-driving behaviors at intersections, aims to increase the computational efficiency of the low-level planner and to enable human-f…
Recent advances in contextual bandit optimization and reinforcement learning have garnered interest in applying these methods to real-world sequential decision making problems. Real-world applications frequently have constraints with respect to a currently deployed policy. Many of the existing constraint-aware algorithms consider problems with a single objective (the reward) and a constraint on the reward with respect to a baseline policy. However, many important applications involve multiple competing objectives and auxiliary constraints. In this paper, we propose a novel Thompson sampling al…
Center of Ubiquitous Computing（普适计算中心）；Faculty of Information Technology and Electrical Engineering（信息技术与电气工程学院）；University of Oulu（奥卢大学）；Politecnico di Milano（米兰理工大学）
In this paper, a robot navigating an environment shared with humans is considered, and a cost function that can be exploited in $\text{RRT}^\text{X}$, a randomized sampling-based replanning algorithm that guarantees asymptotic optimality, to allow for a safe motion is proposed. The cost function is a path length weighted by a danger index based on a prediction of human motion performed using either a linear stochastic model, assuming constant longitudinal velocity and varying lateral velocity, and a GMM/GMR-based model, computed on an experimental dataset of human trajectories. The proposed ap…
Navigating Robot Swarm Through a Virtual Tube with Flow-Adaptive Distribution Control
School of Mathematical Sciences, Beihang University（北京航空航天大学数学科学学院）；School of Automation Science and Electrical Engineering, Beihang University（北京航空航天大学自动化科学与电气工程学院）
In recent years, great efforts have been devoted to deep imitation learning for autonomous driving control, where raw sensory inputs are directly mapped to control actions. However, navigating through densely populated intersections remains a challenging task due to uncertainty caused by uncertain traffic participants. We focus on autonomous navigation at crowded intersections that require interaction with pedestrians. A multi-task conditional imitation learning framework is proposed to adapt both lateral and longitudinal control tasks for safe and efficient interaction. A new benchmark called…
Jet Propulsion Laboratory, California Institute of Technology（加州理工学院喷气推进实验室）；Manipal Institute of Technology（马尼帕尔理工学院）；California Institute of Technology（加州理工学院）；Massachusetts Institute of Technology（麻省理工学院）
We present MLNav, a learning-enhanced path planning framework for safety-critical and resource-limited systems operating in complex environments, such as rovers navigating on Mars. MLNav makes judicious use of machine learning to enhance the efficiency of path planning while fully respecting safety constraints. In particular, the dominant computational cost in such safety-critical settings is running a model-based safety checker on the proposed paths. Our learned search heuristic can simultaneously predict the feasibility for all path options in a single run, and the model-based safety checker…
Johns Hopkins University（约翰斯·霍普金斯大学）；Wilmer Eye Institute at the Johns Hopkins Hospital（约翰斯·霍普金斯医院威尔默眼科研究所）
A fundamental challenge in retinal surgery is safely navigating a surgical tool to a desired goal position on the retinal surface while avoiding damage to surrounding tissues, a procedure that typically requires tens-of-microns accuracy. In practice, the surgeon relies on depth-estimation skills to localize the tool-tip with respect to the retina in order to perform the tool-navigation task, which can be prone to human error. To alleviate such uncertainty, prior work has introduced ways to assist the surgeon by estimating the tool-tip distance to the retina and providing haptic or auditory fee…
We propose GameOpt: a novel hybrid approach to cooperative intersection control for dynamic, multi-lane, unsignalized intersections. Safely navigating these complex and accident prone intersections requires simultaneous trajectory planning and negotiation among drivers. GameOpt is a hybrid formulation that first uses an auction mechanism to generate a priority entrance sequence for every agent, followed by an optimization-based trajectory planner that computes velocity controls that satisfy the priority sequence. This coupling operates at real-time speeds of less than 10 milliseconds in high d…
EC-IoU: Orienting Safety for Object Detectors via Ego-Centric Intersection-over-Union
DENSO AUTOMOTIVE Deutschland GmbH（电装德国汽车有限公司）；Technical University of Munich（慕尼黑工业大学）；Chalmers University of Technology（查尔姆斯理工大学）；University of Gothenburg（哥德堡大学）
Honda Research Institute (HRI) Europe（本田欧洲研究院（HRI Europe））；Eindhoven University of Technology (TU/e)（埃因霍温理工大学（TU/e））
We consider the problem of intelligently navigating through complex traffic. Urban situations are defined by the underlying map structure and special regulatory objects of e.g. a stop line or crosswalk. Thereon dynamic vehicles (cars, bicycles, etc.) move forward, while trying to keep accident risks low. Especially at intersections, the combination and interaction of traffic elements is diverse and human drivers need to focus on specific elements which are critical for their behavior. To support the analysis, we present in this paper the so-called Risk Navigation System (RNS). RNS leverages a …
