For an autonomous vehicle, situation understand-ing is a key capability towards safe and comfortable decision-making and navigation. Information is in general provided bymultiple sources. Prior information about the road topology andtraffic laws can be given by a High Definition (HD) map whilethe perception system provides the description of the spaceand of road entities evolving in the vehicle surroundings. Incomplex situations such as those encountered in urban areas,the road user behaviors are governed by strong interactionswith the others, and with the road network. In such situations,reliable situation understanding is therefore mandatory to avoidinappropriate decisions. Nevertheless, situation understandingis a complex task that requires access to a consistent andnon-misleading representation of the vehicle surroundings. Thispaper proposes a formalism (an interaction lane grid) whichallows to represent, with different levels of abstraction, thenavigable and interacting spaces which must be considered forsafe navigation. A top-down approach is chosen to assess andcharacterize the relevant information of the situation. On a highlevel of abstraction, the identification of the areas of interestwhere the vehicle should pay attention is depicted. On a lowerlevel, it enables to characterize the spatial information in aunified representation and to infer additional information inoccluded areas by reasoning with dynamic objects.
This paper presents a noval method that generates optimal trajectories for autonomous vehicles for in-lane driving scenarios. The method computes a trajectory using a two-phase optimization procedure. In the first phase, the optimization procedure generates a close-form driving guide line with differetiable curvatures. In the second phase, the procedure takes the driving guide line as input, and outputs dynamically feasible, jerk and time optimal trajectories for vehicles driving along the guide line. This method is especially useful for generating trajectories at curvy road where the vehicles…
Understanding human driving behavior is crucial to develop autonomous vehicles' algorithms. However, most low level automation, such as the one in advanced driving assistance systems (ADAS), is based on objective safety measures, which are not always aligned with what the drivers perceive as safe and their correspondent driving behavior. Finding the bridge between the subjective perception and objective safety measures has been analyzed in this paper focusing specifically on lane-change scenarios. Results showed statistically significant differences between what is perceived as safe by drivers…
A Learning-based Discretionary Lane-Change Decision-Making Model with Driving Style Awareness
City University of Hong Kong（香港城市大学）；Zhejiang University of Technology（浙江工业大学）；University of Victoria（维多利亚大学）；The University of Queensland（昆士兰大学）；University of Puerto Rico（波多黎各大学）
National Institute of Informatics（国立情报学研究所）；Carnegie Mellon University（卡内基梅隆大学）；General Motors Company（通用汽车公司）
Connected and Automated Vehicles (CAVs) have real-time information from the surrounding environment by using local on-board sensors, V2X (Vehicle-to-Everything) communications, pre-loaded vehicle-specific lookup tables, and map database. CAVs are capable of improving energy efficiency by incorporating these information. In particular, Eco-Cruise and Eco-Lane Selection on highways and/or motorways have immense potential to save energy, because there are generally fewer traffic controllers and the vehicles keep moving in general. In this paper, we present a cooperative and energy-efficient lane-…
In this paper we present a novel approach for lane detection and segmentation using generative models. Traditionally discriminative models have been employed to classify pixels semantically on a road. We model the probability distribution of lanes and road symbols by training a generative adversarial network. Based on the learned probability distribution, context-aware lanes and road signs are generated for a given image which are further quantized for nearest class label. Proposed method has been tested on BDD100K and Baidu ApolloScape datasets and performs better than state of the art and ex…
We present an approach towards robust lane tracking for assisted and autonomous driving, particularly under poor visibility. Autonomous detection of lane markers improves road safety, and purely visual tracking is desirable for widespread vehicle compatibility and reducing sensor intrusion, cost, and energy consumption. However, visual approaches are often ineffective because of a number of factors, including but not limited to occlusion, poor weather conditions, and paint wear-off. Our method, named SafeDrive, attempts to improve visual lane detection approaches in drastically degraded visual…
University of Freiburg（弗赖堡大学）；Woven by Toyota（丰田编织公司）；University of Technology Nuremberg（纽伦堡工业大学）
Lane graph estimation is an essential and highly challenging task in automated driving and HD map learning. Existing methods using either onboard or aerial imagery struggle with complex lane topologies, out-of-distribution scenarios, or significant occlusions in the image space. Moreover, merging overlapping lane graphs to obtain consistent large-scale graphs remains difficult. To overcome these challenges, we propose a novel bottom-up approach to lane graph estimation from aerial imagery that aggregates multiple overlapping graphs into a single consistent graph. Due to its modular design, our…
The ability to estimate human intentions and interact with human drivers intelligently is crucial for autonomous vehicles to successfully achieve their objectives. In this paper, we propose a game theoretic planning algorithm that models human opponents with an iterative reasoning framework and estimates human latent cognitive states through probabilistic inference and active learning. By modeling the interaction as a partially observable Markov decision process with adaptive state and action spaces, our algorithm is able to accomplish real-time lane changing tasks in a realistic driving simul…
Simulation has long been an essential part of testing autonomous driving systems, but only recently has simulation been useful for building and training self-driving vehicles. Vehicle behavioural models are necessary to simulate the interactions between robot cars. This paper proposed a new method to formalize the lane-changing model in urban driving scenarios. We define human incentives from different perspectives, speed incentive, route change incentive, comfort incentive and courtesy incentive etc. We applied a decision-theoretical tool, called Multi-Criteria Decision Making (MCDM) to take …
Institute of Industrial Science, The University of Tokyo（东京大学工业科学研究所）；Hino Motors, Ltd.（日野汽车公司）
Lane change is a very demanding driving task and number of traffic accidents are induced by mistaken maneuvers. An automated lane change system has the potential to reduce driver workload and to improve driving safety. One challenge is how to improve driver acceptance on the automated system. From the viewpoint of human factors, an automated system with different styles would improve user acceptance as the drivers can adapt the style to different driving situations. This paper proposes a method to design different lane change styles in automated driving by analysis and modeling of truck driver…
Predicting a vehicle's trajectory is an essential ability for autonomous vehicles navigating through complex urban traffic scenes. Bird's-eye-view roadmap information provides valuable information for making trajectory predictions, and while state-of-the-art models extract this information via image convolution, auxiliary loss functions can augment patterns inferred from deep learning by further encoding common knowledge of social and legal driving behaviors. Since human driving behavior is inherently multimodal, models which allow for multimodal output tend to outperform single-prediction mod…
Machine Learning-Based Vehicle Intention Trajectory Recognition and Prediction for Autonomous Driving
University of Southern California（南加州大学）；Virginia Tech（弗吉尼亚理工大学）；Miami University（迈阿密大学）；Northern Arizona University（北亚利桑那大学）；Northeastern University（东北大学）
Developing, Analyzing, and Evaluating Vehicular Lane Keeping Algorithms Under Dynamic Lighting and Weather Conditions Using Electric Vehicles
Rice University（莱斯大学）；Pennsylvania State University（宾夕法尼亚州立大学）；Willamette University（威拉姆特大学）；Louisiana State University（路易斯安那州立大学）；Michigan State University（密歇根州立大学）；Lawrence Technological University（劳伦斯理工大学）
Jaguar Land Rover Limited（捷豹路虎有限公司）；University of Surrey（萨里大学）
Quantifying and encoding occupants' preferences as an objective function for the tactical decision making of autonomous vehicles is a challenging task. This paper presents a low-complexity approach for lane-change initiation and planning to facilitate highly automated driving on freeways. Conditions under which human drivers find different manoeuvres desirable are learned from naturalistic driving data, eliminating the need for an engineered objective function and incorporation of expert knowledge in form of rules. Motion planning is formulated as a finite-horizon optimisation problem with saf…
Beijing Institute of Technology（北京理工大学）；California PATH, University of California, Berkeley（加州大学伯克利分校加州PATH项目）
We apply Deep Q-network (DQN) with the consideration of safety during the task for deciding whether to conduct the maneuver. Furthermore, we design two similar Deep Q learning frameworks with quadratic approximator for deciding how to select a comfortable gap and just follow the preceding vehicle. Finally, a polynomial lane change trajectory is generated and Pure Pursuit Control is implemented for path tracking. We demonstrate the effectiveness of this framework in simulation, from both the decision-making and control layers. The proposed architecture also has the potential to be extended to o…
National Chiao Tung University（台湾交通大学）；National Tsing Hua University（台湾清华大学）
Inspired by the UNet architecture of semantic image segmentation, we propose a lightweight UNet using depthwise separable convolutions (DSUNet) for end-to-end learning of lane detection and path prediction (PP) in autonomous driving. We also design and integrate a PP algorithm with convolutional neural network (CNN) to form a simulation model (CNN-PP) that can be used to assess CNN's performance qualitatively, quantitatively, and dynamically in a host agent car driving along with other agents all in a real-time autonomous manner. DSUNet is 5.16x lighter in model size and 1.61x faster in infere…
Nanyang Technological University（南洋理工大学）；Energy Research Institute, Nanyang Technological University（南洋理工大学能源研究院）
Decision-making is critical for lane change in autonomous driving. Reinforcement learning (RL) algorithms aim to identify the values of behaviors in various situations and thus they become a promising pathway to address the decision-making problem. However, poor runtime safety hinders RL-based decision-making strategies from complex driving tasks in practice. To address this problem, human demonstrations are incorporated into the RL-based decision-making strategy in this paper. Decisions made by human subjects in a driving simulator are treated as safe demonstrations, which are stored into the…
Michigan State University（密歇根州立大学）；Clemson University（克莱姆森大学）
It is expected that many human drivers will still prefer to drive themselves even if the self-driving technologies are ready. Therefore, human-driven vehicles and autonomous vehicles (AVs) will coexist in a mixed traffic for a long time. To enable AVs to safely and efficiently maneuver in this mixed traffic, it is critical that the AVs can understand how humans cope with risks and make driving-related decisions. On the other hand, the driving environment is highly dynamic and ever-changing, and it is thus difficult to enumerate all the scenarios and hard-code the controllers. To face up these …
LANet: A Lane Boundaries-Aware Approach For Robust Trajectory Prediction
Artificial Intelligence Creative Research Lab, ETRI University of Science（ETRI人工智能创意研究实验室，科学技术大学）；Artificial Intelligence Creative Research Lab, ETRI Daejeon, South Korea（ETRI人工智能创意研究实验室，韩国大田）
Self-supervised learning (SSL) is an emerging technique that has been successfully employed to train convolutional neural networks (CNNs) and graph neural networks (GNNs) for more transferable, generalizable, and robust representation learning. However its potential in motion forecasting for autonomous driving has rarely been explored. In this study, we report the first systematic exploration and assessment of incorporating self-supervision into motion forecasting. We first propose to investigate four novel self-supervised learning tasks for motion forecasting with theoretical rationale and qu…
Autonomous driving promises to transform road transport. Multi-vehicle and multi-lane scenarios, however, present unique challenges due to constrained navigation and unpredictable vehicle interactions. Learning-based methods---such as deep reinforcement learning---are emerging as a promising approach to automatically design intelligent driving policies that can cope with these challenges. Yet, the process of safely learning multi-vehicle driving behaviours is hard: while collisions---and their near-avoidance---are essential to the learning process, directly executing immature policies on auton…
Neural network-based driving planners have shown great promises in improving task performance of autonomous driving. However, it is critical and yet very challenging to ensure the safety of systems with neural network based components, especially in dense and highly interactive traffic environments. In this work, we propose a safety-driven interactive planning framework for neural network-based lane changing. To prevent over conservative planning, we identify the driving behavior of surrounding vehicles and assess their aggressiveness, and then adapt the planned trajectory for the ego vehicle …
Distributional Soft Actor-Critic with Harmonic Gradient for Safe and Efficient Autonomous Driving in Multi-lane Scenarios
School of Vehicle and Mobility, Tsinghua University（清华大学车辆与移动系统学院）；School of Mechanical Engineering, University of Science and Technology Beijing（北京科技大学机械工程学院）；College of Artificial Intelligence, Tsinghua University（清华大学人工智能学院）
Autonomous systems require identifying the environment and it has a long way to go before putting it safely into practice. In autonomous driving systems, the detection of obstacles and traffic lights are of importance as well as lane tracking. In this study, an autonomous driving system is developed and tested in the experimental environment designed for this purpose. In this system, a model vehicle having a camera is used to trace the lanes and avoid obstacles to experimentally study autonomous driving behavior. Convolutional Neural Network models were trained for Lane tracking. For the vehic…
Technical University of Munich（慕尼黑工业大学）；Carnegie Mellon University（卡内基梅隆大学）
Recently, autonomous driving has made substantial progress in addressing the most common traffic scenarios like intersection navigation and lane changing. However, most of these successes have been limited to scenarios with well-defined traffic rules and require minimal negotiation with other vehicles. In this paper, we introduce a previously unconsidered, yet everyday, high-conflict driving scenario requiring negotiations between agents of equal rights and priorities. There exists no centralized control structure and we do not allow communications. Therefore, it is unknown if other drivers ar…
Tsinghua University（清华大学）；National University of Singapore（新加坡国立大学）；China Agricultural University（中国农业大学）
In this paper, we propose a new reinforcement learning (RL) algorithm, called encoding distributional soft actor-critic (E-DSAC), for decision-making in autonomous driving. Unlike existing RL-based decision-making methods, E-DSAC is suitable for situations where the number of surrounding vehicles is variable and eliminates the requirement for manually pre-designed sorting rules, resulting in higher policy performance and generality. We first develop an encoding distributional policy iteration (DPI) framework by embedding a permutation invariant module, which employs a feature neural network (N…
Generation of Training Data from HD Maps in the Lanelet2 Framework
FZI Research Center for Information Technology（FZI信息技术研究中心）；Institute of Measurement and Control Systems, Karlsruhe Institute of Technology (KIT)（卡尔斯鲁厄理工学院测量与控制系统研究所）
Uber Advanced Technologies Group（优步先进技术集团）；University of Toronto（多伦多大学）；MIT（麻省理工学院）
One of the fundamental challenges to scale self-driving is being able to create accurate high definition maps (HD maps) with low cost. Current attempts to automate this process typically focus on simple scenarios, estimate independent maps per frame or do not have the level of precision required by modern self driving vehicles. In contrast, in this paper we focus on drawing the lane boundaries of complex highways with many lanes that contain topology changes due to forks and merges. Towards this goal, we formulate the problem as inference in a directed acyclic graphical model (DAG), where the …
VNU University of Engineering and Technology（越南国立大学工程与技术大学）
This paper addresses the problem of lane detection which is fundamental for self-driving vehicles. Our approach exploits both colour and depth information recorded by a single RGB-D camera to better deal with negative factors such as lighting conditions and lane-like objects. In the approach, colour and depth images are first converted to a half-binary format and a 2D matrix of 3D points. They are then used as the inputs of template matching and geometric feature extraction processes to form a response map so that its values represent the probability of pixels being lane markers. To further im…
Efficient and Safe Planner for Automated Driving on Ramps Considering Unsatisfication
Department of Computer Science, University of Liverpool（利物浦大学计算机科学系）；James Watt School of Engineering, University of Glasgow（格拉斯哥大学詹姆斯·瓦特工程学院）；School of Engineering, Cardiff University（卡迪夫大学工程学院）；School of Computing Science, University of Glasgow（格拉斯哥大学计算机科学学院）
Vision-based DRL Autonomous Driving Agent with Sim2Real Transfer
Chair of Econometrics and Statistics, esp. in the Transport Sector, Technische Universität Dresden（德累斯顿工业大学交通领域计量经济学与统计学讲席）；Center for Scalable Data Analytics and Artificial Intelligence (ScaDS.AI) Dresden/Leipzig（德累斯顿/莱ipzig可扩展数据分析与人工智能中心（ScaDS.AI））
University of Michigan（密歇根大学）；Toyota Motor North America Research and Development（丰田北美研发公司）
In the event of sensor failure, autonomous vehicles need to safely execute emergency maneuvers while avoiding other vehicles on the road. To accomplish this, the sensor-failed vehicle must predict the future semantic behaviors of other drivers, such as lane changes, as well as their future trajectories given a recent window of past sensor observations. We address the first issue of semantic behavior prediction in this paper, which is a precursor to trajectory prediction, by introducing a framework that leverages the power of recurrent neural networks (RNNs) and graphical models. Our goal is to…
Institute of Power Machinery and Vehicular Engineering, College of Energy Engineering, Zhejiang University（浙江大学能源工程学院动力机械及车辆工程研究所）
An automated driving system should have the ability to supervise its own performance and to request human driver to take over when necessary. In the lane keeping scenario, the prediction of vehicle future trajectory is the key to realize safe and trustworthy driving automation. Previous studies on vehicle trajectory prediction mainly fall into two categories, i.e. physics-based and manoeuvre-based methods. Using a physics-based methodology, this paper proposes a lane departure prediction algorithm based on closed-loop vehicle dynamics model. We use extended Kalman filter to estimate the curren…
Graz University of Technology（格拉茨技术大学）；MAGNA Steyr Fahrzeugtechnik AG & CO KG（麦格纳斯太尔汽车技术股份两合公司）
Virtual development and prototyping has already become an integral part in the field of automated driving systems (ADS). There are plenty of software tools that are used for the virtual development of ADS. One such tool is CarMaker from IPG Automotive, which is widely used in the scientific community and in the automotive industry. It offers a broad spectrum of implementation and modelling possibilities of the vehicle, driver behavior, control, sensors, and environmental models. Focusing on the virtual development of highly automated driving functions on the vehicle guidance level, it is essen…
Accurately forecasting the future movements of surrounding vehicles is essential for safe and efficient operations of autonomous driving cars. This task is difficult because a vehicle's moving trajectory is greatly determined by its driver's intention, which is often hard to estimate. By leveraging attention mechanisms along with long short-term memory (LSTM) networks, this work learns the relation between a driver's intention and the vehicle's changing positions relative to road infrastructures, and uses it to guide the prediction. Different from other state-of-the-art solutions, our work tre…
Institute of Automation, Chinese Academy of Sciences（中国科学院自动化研究所）；University of Chinese Academy of Sciences（中国科学院大学）
Autonomous driving decision-making is a great challenge due to the complexity and uncertainty of the traffic environment. Combined with the rule-based constraints, a Deep Q-Network (DQN) based method is applied for autonomous driving lane change decision-making task in this study. Through the combination of high-level lateral decision-making and low-level rule-based trajectory modification, a safe and efficient lane change behavior can be achieved. With the setting of our state representation and reward function, the trained agent is able to take appropriate actions in a real-world-like simula…
mercedes-Benz AG（梅赛德斯-奔驰股份公司）；Ulm University（乌尔姆大学）；Eberhard Karls University Tübingen（埃伯哈德·卡尔斯大学图宾根）
The prediction of surrounding traffic participants behavior is a crucial and challenging task for driver assistance and autonomous driving systems. Today's approaches mainly focus on modeling dynamic aspects of the traffic situation and try to predict traffic participants behavior based on this. In this article we take a first step towards extending this common practice by calculating location-specific a-priori lane change probabilities. The idea behind this is straight forward: The driving behavior of humans may vary in exactly the same traffic situation depending on the respective location. …
Autonomous driving is a challenging domain that entails multiple aspects: a vehicle should be able to drive to its destination as fast as possible while avoiding collision, obeying traffic rules and ensuring the comfort of passengers. In this paper, we present a deep learning variant of thresholded lexicographic Q-learning for the task of urban driving. Our multi-objective DQN agent learns to drive on multi-lane roads and intersections, yielding and changing lanes according to traffic rules. We also propose an extension for factored Markov Decision Processes to the DQN architecture that provid…
This paper proposes a framework to recognize driving intentions and to predict driving behaviors of lane changing on the highway by using externally sensable traffic data from the host-vehicle. The framework consists of a driving characteristic estimator and a driving behavior predictor. A driver's implicit driving characteristic information is uniquely determined and detected by proposed the online-estimator. Neural-network based behavior predictor is developed and validated by testing with the real naturalistic traffic data from Next Generation Simulation (NGSIM), which demonstrates the effe…
For lateral vehicle dynamics, planning trajectories for lane-keeping and lane-change can be generalized as a path planning task to stabilize a vehicle onto a target lane, which is a fundamental element in nowadays autonomous driving systems. On the other hand, two-point steering for lane-change and lane-keeping has been investigated by researchers from psychology as a sensorimotor mechanism of human drivers. In the first part of this paper, using knowledge of geometric control, we will first design a path planner which satisfies five design objectives: generalization for different vehicle mode…
University of Freiburg（弗莱堡大学）；Toyota Research Institute（丰田研究所）
Lane-level scene annotations provide invaluable data in autonomous vehicles for trajectory planning in complex environments such as urban areas and cities. However, obtaining such data is time-consuming and expensive since lane annotations have to be annotated manually by humans and are as such hard to scale to large areas. In this work, we propose a novel approach for lane geometry estimation from bird's-eye-view images. We formulate the problem of lane shape and lane connections estimation as a graph estimation problem where lane anchor points are graph nodes and lane segments are graph edge…
