This paper investigates a hybrid solution which combines deep reinforcement learning (RL) and classical trajectory planning for the following in front application. Here, an autonomous robot aims to stay ahead of a person as the person freely walks around. Following in front is a challenging problem as the user's intended trajectory is unknown and needs to be estimated, explicitly or implicitly, by the robot. In addition, the robot needs to find a feasible way to safely navigate ahead of human trajectory. Our deep RL module implicitly estimates human trajectory and produces short-term navigational goals to guide the robot. These goals are used by a trajectory planner to smoothly navigate the robot to the short-term goals, and eventually in front of the user. We employ curriculum learning in the deep RL module to efficiently achieve a high return. Our system outperforms the state-of-the-art in following ahead and is more reliable compared to end-to-end alternatives in both the simulation and real world experiments. In contrast to a pure deep RL approach, we demonstrate zero-shot transfer of the trained policy from simulation to the real world.
Mody Institute of Technology and Science（莫迪理工学院）；PEC University of Technology（旁遮普工程学院大学）
The paper presents the electronic design and motion planning of a robot based on decision making regarding its straight motion and precise turn using Artificial Neural Network (ANN). The ANN helps in learning of robot so that it performs motion autonomously. The weights calculated are implemented in microcontroller. The performance has been tested to be excellent.
Path planning is one of the most vital elements of mobile robotics. With a priori knowledge of the environment, global path planning provides a collision-free route through the workspace. The global path plan can be calculated with a variety of informed search algorithms, most notably the A* search method, guaranteed to deliver a complete and optimal solution that minimizes the path cost. Path planning optimization typically looks to minimize the distance traversed from start to goal, yet many mobile robot applications call for additional path planning objectives, presenting a multiobjective o…
Volvo Group（沃尔沃集团）；Chalmers University of Technology（查尔姆斯理工大学）；University of Illinois at Urbana-Champaign（伊利诺伊大学厄巴纳-香槟分校）；Stanford University（斯坦福大学）
Tactical decision making for autonomous driving is challenging due to the diversity of environments, the uncertainty in the sensor information, and the complex interaction with other road users. This paper introduces a general framework for tactical decision making, which combines the concepts of planning and learning, in the form of Monte Carlo tree search and deep reinforcement learning. The method is based on the AlphaGo Zero algorithm, which is extended to a domain with a continuous state space where self-play cannot be used. The framework is applied to two different highway driving cases …
This paper presents a hybrid motion planning strategy that combines a deep generative network with a conventional motion planning method. Existing planning methods such as A* and Hybrid A* are widely used in path planning tasks because of their ability to determine feasible paths even in complex environments; however, they have limitations in terms of efficiency. To overcome these limitations, a path planning algorithm based on a neural network, namely the neural Hybrid A*, is introduced. This paper proposes using a conditional variational autoencoder (CVAE) to guide the search algorithm by ex…
National Institute of Informatics（国立信息学研究所）；Hunan University（湖南大学）
In the path planning problem of autonomous application, the existing studies separately consider the path planning and trajectory tracking control of the autonomous vehicle and few of them have integrated the trajectory planning and trajectory control together. To fill in this research gap, this study proposes an integrated trajectory planning and trajectory control method. This paper also studies the collision avoidance problem of autonomous by considering static and dynamic obstacles. Simulation results have been presented to show the effectiveness of the proposed control method.
This paper presents a novel data-driven approach to vehicle motion planning and control in off-road driving scenarios. For autonomous off-road driving, environmental conditions impact terrain traversability as a function of weather, surface composition, and slope. Geographical information system (GIS) and National Centers for Environmental Information datasets are processed to provide this information for interactive planning and control system elements. A top-level global route planner (GRP) defines optimal waypoints using dynamic programming (DP). A local path planner (LPP) computes a desire…
Residual Chain Prediction for Autonomous Driving Path Planning
Technical University of Munich（慕尼黑工业大学）；School of Computer Science, Nanjing University of Information Science & Technology（南京信息工程大学计算机学院）；School of Computer and Information Engineering, Fuyang Normal University（阜阳师范大学计算机与信息工程学院）
针对端到端自动驾驶路径规划中行为克隆因曼哈顿距离产生协变量偏移的问题，提出 Residual Chain Loss 动态调整损失以增强时序依赖，并在 nuScenes 上验证其性能提升。
TuSimple Inc.（图森未来公司）；Georgia Institute of Technology（佐治亚理工学院）；University of California, Berkeley（加州大学伯克利分校）；California PATH（加州先进交通与高速公路合作伙伴项目）
In this survey, we systematically summarize the current literature on studies that apply reinforcement learning (RL) to the motion planning and control of autonomous vehicles. Many existing contributions can be attributed to the pipeline approach, which consists of many hand-crafted modules, each with a functionality selected for the ease of human interpretation. However, this approach does not automatically guarantee maximal performance due to the lack of a system-level optimization. Therefore, this paper also presents a growing trend of work that falls into the end-to-end approach, which typ…
HyPlan: Hybrid Learning-Assisted Planning Under Uncertainty for Safe Autonomous Driving
Saarland University, Computer Science Dept.（萨尔兰大学计算机科学系）；German Research Center for Artificial Intelligence (DFKI)（德国人工智能研究中心）；French National Centre for Scientific Research (CNRS)（法国国家科学研究中心）
NATO STO Centre for Maritime Research and Experimentation (CMRE)（北约科技组织海事研究与实验中心）
In this paper we address the mine countermeasures (MCM) search problem for an autonomous underwater vehicle (AUV) surveying the seabed using a side-looking sonar. We propose a coverage path planning method that adapts the AUV track spacing with the objective of collecting better data. We achieve this by shifting the coverage overlap at the tail of the sensor range where the lowest data quality is expected. To assess the algorithm, we collected data from three at-sea experiments. The adaptive survey allowed the AUV to recover from a situation where the sensor range was overestimated and resulte…
In this work, we propose a hierarchical reinforcement learning (HRL) structure which is capable of performing autonomous vehicle planning tasks in simulated environments with multiple sub-goals. In this hierarchical structure, the network is capable of 1) learning one task with multiple sub-goals simultaneously; 2) extracting attentions of states according to changing sub-goals during the learning process; 3) reusing the well-trained network of sub-goals for other similar tasks with the same sub-goals. The states are defined as processed observations which are transmitted from the perception s…
Technion - Israel Institute of Technology（以色列理工学院）
An important capability of autonomous Unmanned Aerial Vehicles (UAVs) is autonomous landing while avoiding collision with obstacles in the process. Such capability requires real-time local trajectory planning. Although trajectory-planning methods have been introduced for cases such as emergency landing, they have not been evaluated in real-life scenarios where only the surface of obstacles can be sensed and detected. We propose a novel optimization framework using a pre-planned global path and a priority map of the landing area. Several trajectory planning algorithms were implemented and evalu…
Shanghai Automotive Industry Corporation, Ltd.（上海汽车工业股份有限公司）
This paper presents an automatic merging algorithm for autonomous driving vehicles, which decouples the specific motion planning problem into a Dual-Layer Automatic Merge Planning (DL_AMP) and a Descent-Based Trajectory Optimization (DBTO). This work leads to great improvements in finding the best merge opportunity, lateral and longitudinal merge planning and control, trajectory postprocessing and driving comfort.
Collision-tolerant trajectory planning is the consideration that collisions, if they are planned appropriately, enable more effective path planning for robots capable of handling them. A mixed integer programming (MIP) optimization formulation demonstrates the computational practicality of optimizing trajectories that comprise planned collisions. A damage quantification function is proposed, and the influence of damage functions constraints on the trajectory are studied in simulation. Using a simple example, an increase in performance is achieved under this schema as compared to collision-free…
SRM Institute of Science and Technology（SRM科学技术研究所）
Autonomous Vehicles are currently being tested in a variety of scenarios. As we move towards Autonomous Vehicles, how should intersections look? To answer that question, we break down an intersection management into the different conundrums and scenarios involved in the trajectory planning and current approaches to solve them. Then, a brief analysis of current works in autonomous intersection is conducted. With a critical eye, we try to delve into the discrepancies of existing solutions while presenting some critical and important factors that have been addressed. Furthermore, open issues that…
Path planning for autonomous driving with dynamic obstacles poses a challenge because it needs to perform a higher-dimensional search (with time-dimension) while still meeting real-time constraints. This paper proposes an algorithm-hardware co-optimization approach to accelerate path planning with high-dimensional search space. First, we reduce the time for a nearest neighbor search and collision detection by mapping nodes and obstacles to a lower-dimensional space and memoizing recent search results. Then, we propose a hardware extension for efficient memoization. The experimental results on …
University of Edinburgh（爱丁堡大学）；Five AI Ltd.（Five AI有限公司）
Inscrutable AI systems are difficult to trust, especially if they operate in safety-critical settings like autonomous driving. Therefore, there is a need to build transparent and queryable systems to increase trust levels. We propose a transparent, human-centric explanation generation method for autonomous vehicle motion planning and prediction based on an existing white-box system called IGP2. Our method integrates Bayesian networks with context-free generative rules and can give causal natural language explanations for the high-level driving behaviour of autonomous vehicles. Preliminary test…
fortiss GmbH（福蒂斯有限公司）；Technische Universität München（慕尼黑工业大学）
Motion planning for urban environments with numerous moving agents can be viewed as a combinatorial problem. With passing an obstacle before, after, right or left, there are multiple options an autonomous vehicle could choose to execute. These combinatorial aspects need to be taken into account in the planning framework. We address this problem by proposing a novel planning approach that combines trajectory planning and maneuver reasoning. We define a classification for dynamic obstacles along a reference curve that allows us to extract tactical decision sequences. We separate longitudinal and…
Planning by Simulation: Motion Planning with Learning-based Parallel Scenario Prediction for Autonomous Driving
Faculty of Electrical Engineering, Mathematics and Computer Science, University of Twente（代尔夫特理工大学电子工程、数学与计算机科学学院）；Department of Electrical Engineering, Wright State University（威斯汀大学电气工程系）
University of California at Berkeley（加州大学伯克利分校）；California PATH, University of California at Berkeley（加州大学伯克利分校加州PATH）
Autonomous cars have to navigate in dynamic environment which can be full of uncertainties. The uncertainties can come either from sensor limitations such as occlusions and limited sensor range, or from probabilistic prediction of other road participants, or from unknown social behavior in a new area. To safely and efficiently drive in the presence of these uncertainties, the decision-making and planning modules of autonomous cars should intelligently utilize all available information and appropriately tackle the uncertainties so that proper driving strategies can be generated. In this paper, …
Decision-Making-Based Path Planning for Autonomous UAVs: A Survey
Department of Cybernetics, Faculty of Electrical Engineering, Czech Technical University in Prague（控制系，电气工程系，布拉格捷克技术大学）；Department of Computer, Federal University of São Carlos（计算机系，萨o卡洛斯联邦大学）
Implicit neural representations have shown promising potential for the 3D scene reconstruction. Recent work applies it to autonomous 3D reconstruction by learning information gain for view path planning. Effective as it is, the computation of the information gain is expensive, and compared with that using volumetric representations, collision checking using the implicit representation for a 3D point is much slower. In the paper, we propose to 1) leverage a neural network as an implicit function approximator for the information gain field and 2) combine the implicit fine-grained representation …
This paper discusses opportunities to parallelize graph based path planning algorithms in a time varying environment. Parallel architectures have become commonplace, requiring algorithm to be parallelized for efficient execution. An additional focal point of this paper is the inclusion of inaccuracies in path planning as a result of forecast error variance, accuracy of calculation in the cost functions and a different observed vehicle speed in the real mission than planned. In this context, robust path planning algorithms will be described. These algorithms are equally applicable to land based…
The shortage of workforce and increasing cost of maintenance has forced many farm industrialists to shift towards automated and mechanized approaches. The key component for autonomous systems is the path planning techniques used. Coverage path planning (CPP) algorithm is used for navigating over farmlands to perform various agricultural operations such as seeding, ploughing, or spraying pesticides and fertilizers. This report paper compares novel algorithms for autonomous navigation of farmlands. For reduction of navigational constraints, a high-resolution grid map representation is taken into…
Hong Kong University of Science and Technology（香港科技大学）
Real-time kinodynamic trajectory planning in dynamic environments is critical yet challenging for autonomous driving. In this letter, we propose an efficient trajectory planning system for autonomous driving in complex dynamic scenarios through iterative and incremental path-speed optimization. Exploiting the decoupled structure of the planning problem, a path planner based on Gaussian process first generates a continuous arc-length parameterized path in the Frenét frame, considering static obstacle avoidance and curvature constraints. We theoretically prove that it is a good generalization of…
Stanford University（斯坦福大学）；Honda Research Institute（本田研究所）
Urban intersections represent a complex environment for autonomous vehicles with many sources of uncertainty. The vehicle must plan in a stochastic environment with potentially rapid changes in driver behavior. Providing an efficient strategy to navigate through urban intersections is a difficult task. This paper frames the problem of navigating unsignalized intersections as a partially observable Markov decision process (POMDP) and solves it using a Monte Carlo sampling method. Empirical results in simulation show that the resulting policy outperforms a threshold-based heuristic strategy on s…
Motion planning under uncertainty for an autonomous system can be formulated as a Markov Decision Process with a continuous state space. In this paper, we propose a novel solution to this decision-theoretic planning problem that directly obtains the continuous value function with only the first and second moments of the transition probabilities, alleviating the requirement for an explicit transition model in the literature. We achieve this by expressing the value function as a linear combination of basis functions and approximating the Bellman equation by a partial differential equation, where…
University of California, Berkeley（加州大学伯克利分校）；Mitsubishi Electric Research Laboratories（三菱电机研究实验室）
This paper presents an integrated motion planning system for autonomous vehicle (AV) parking in the presence of other moving vehicles. The proposed system includes 1) a hybrid environment predictor that predicts the motions of the surrounding vehicles and 2) a strategic motion planner that reacts to the predictions. The hybrid environment predictor performs short-term predictions via an extended Kalman filter and an adaptive observer. It also combines short-term predictions with a driver behavior cost-map to make long-term predictions. The strategic motion planner comprises 1) a model predicti…
This paper proposes a fast and accurate trajectory planning algorithm for autonomous parking. Nominally, an optimal control problem should be formulated to describe this scheme, but the dimensionality of the optimal control problem is usually large, because the vehicle needs to avoid collision with every obstacle at every moment during the entire dynamic process. Although an initial guess obtained by a sample-and-search based planner facilitates the numerical optimization process, it is still far from being as fast as real-time. To address this issue, we replace all of the collision-avoidance …
ETH Zurich（苏黎世联邦理工学院）；Imperial College London（帝国理工学院）
Exploration and collision-free navigation through an unknown environment is a fundamental task for autonomous robots. In this paper, a novel exploration strategy for Micro Aerial Vehicles (MAVs) is presented. The goal of the exploration strategy is the reduction of map entropy regarding occupancy probabilities, which is reflected in a utility function to be maximised. We achieve fast and efficient exploration performance with tight integration between our octree-based occupancy mapping approach, frontier extraction, and motion planning-as a hybrid between frontier-based and sampling-based expl…
The land used for grazing cattle takes up about one-third of the land in the United States. These areas can be highly rugged. Yet, they need to be maintained to prevent weeds from taking over the nutritious grassland. This can be a daunting task especially in the case of organic farming since herbicides cannot be used. In this paper, we present the design of Cowbot, an autonomous weed mowing robot for pastures. Cowbot is an electric mower designed to operate in the rugged environments on cow pastures and provide a cost-effective method for weed control in organic farms. Path planning for the C…
School of Mechanical and Aerospace Engineering, Nanyang Technological University（南洋理工大学机械与宇航工程学院）
This paper presents a novel integrated approach to deal with the decision making and motion planning for lane-change maneuvers of autonomous vehicle (AV) considering social behaviors of surrounding traffic occupants. Reflected by driving styles and intentions of surrounding vehicles, the social behaviors are taken into consideration during the modelling process. Then, the Stackelberg Game theory is applied to solve the decision-making, which is formulated as a non-cooperative game problem. Besides, potential field is adopted in the motion planning model, which uses different potential function…
Zhejiang University（浙江大学）；Shenzhen Sensetime Technology CO.,LTD（深圳商汤科技有限公司）；Shenzhen Institutes of Advanced Technology, Chinese Academy of Sciences（中国科学院深圳先进技术研究院）
Autonomous Underwater Vehicle-Manipulator systems (AUVMS) is a new tool for ocean exploration, the AUVMS path planning problem is addressed in this paper. AUVMS is a high dimension system with a large difference in inertia distribution, also it works in a complex environment with obstacles. By integrating the rapidly-exploring random tree(RRT) algorithm with the AUVMS kinematics model, the proposed RRTAUVMS algorithm could randomly sample in the configuration space(C-Space), and also grow the tree directly towards the workspace goal in the task space. The RRTAUVMS can also deal with the redund…
Birla Institute of Technology and Science, Pilani（皮拉尼比尔拉理工学院）；Cleveland State University（克利夫兰州立大学）；Dajiang Innovation Technology Co., Ltd（大疆创新科技有限公司）；Carnegie Mellon University（卡内基梅隆大学）
Safety-guaranteed motion planning is critical for self-driving cars to generate collision-free trajectories. A layered motion planning approach with decoupled path and speed planning is widely used for this purpose. This approach is prone to be suboptimal in the presence of dynamic obstacles. Spatial-temporal approaches deal with path planning and speed planning simultaneously; however, the existing methods only support simple-shaped corridors like cuboids, which restrict the search space for optimization in complex scenarios. We propose to use trapezoidal prism-shaped corridors for optimizati…
Institute of Automotive Technology, Technical University of Munich（慕尼黑工业大学汽车技术研究所）；Institute for Ethics in Artificial Intelligence, Technical University of Munich（慕尼黑工业大学人工智能伦理研究所）
With the rise of AI and automation, moral decisions are being put into the hands of algorithms that were formerly the preserve of humans. In autonomous driving, a variety of such decisions with ethical implications are made by algorithms for behavior and trajectory planning. Therefore, we present an ethical trajectory planning algorithm with a framework that aims at a fair distribution of risk among road users. Our implementation incorporates a combination of five essential ethical principles: minimization of the overall risk, priority for the worst-off, equal treatment of people, responsibili…
In this paper, a real-time quasi-optimal trajectory planning scheme is employed to guide an autonomous underwater vehicle (AUV) safely into a funnel-shape stationary docking station. By taking advantage of the direct method of calculus of variation and inverse dynamics optimization, the proposed trajectory planner provides a computationally efficient framework for autonomous underwater docking in a 3D cluttered undersea environment. Vehicular constraints, such as constraints on AUV states and actuators; boundary conditions, including initial and final vehicle poses; and environmental constrain…
Algorithms for motion planning in unknown environments are generally limited in their ability to reason about the structure of the unobserved environment. As such, current methods generally navigate unknown environments by relying on heuristic methods to choose intermediate objectives along frontiers. We present a unified method that combines map prediction and motion planning for safe, time-efficient autonomous navigation of unknown environments by dynamically-constrained robots. We propose a data-driven method for predicting the map of the unobserved environment, using the robot's observatio…
