Autonomous vehicles must be comprehensively evaluated before deployed in cities and highways. However, most existing evaluation approaches for autonomous vehicles are static and lack adaptability, so they are usually inefficient in generating challenging scenarios for tested vehicles. In this paper, we propose an adaptive evaluation framework to efficiently evaluate autonomous vehicles in adversarial environments generated by deep reinforcement learning. Considering the multimodal nature of dangerous scenarios, we use ensemble models to represent different local optimums for diversity. We then utilize a nonparametric Bayesian method to cluster the adversarial policies. The proposed method is validated in a typical lane-change scenario that involves frequent interactions between the ego vehicle and the surrounding vehicles. Results show that the adversarial scenarios generated by our method significantly degrade the performance of the tested vehicles. We also illustrate different patterns of generated adversarial environments, which can be used to infer the weaknesses of the tested vehicles.
Accurately forecasting the future movements of surrounding vehicles is essential for safe and efficient operations of autonomous driving cars. This task is difficult because a vehicle's moving trajectory is greatly determined by its driver's intention, which is often hard to estimate. By leveraging attention mechanisms along with long short-term memory (LSTM) networks, this work learns the relation between a driver's intention and the vehicle's changing positions relative to road infrastructures, and uses it to guide the prediction. Different from other state-of-the-art solutions, our work tre…
King’s College London（伦敦国王学院）；Orange Labs Services（奥兰治实验室服务公司）
In this paper, a framework for lane merge coordination is presented utilising a centralised system, for connected vehicles. The delivery of trajectory recommendations to the connected vehicles on the road is based on a Traffic Orchestrator and a Data Fusion as the main components. Deep Reinforcement Learning and data analysis is used to predict trajectory recommendations for connected vehicles, taking into account unconnected vehicles for those suggestions. The results highlight the adaptability of the Traffic Orchestrator, when employing Dueling Deep Q-Network in an unseen real world merging …
University of Michigan（密歇根大学）；Toyota Motor North America Research and Development（丰田北美研发公司）
In the event of sensor failure, autonomous vehicles need to safely execute emergency maneuvers while avoiding other vehicles on the road. To accomplish this, the sensor-failed vehicle must predict the future semantic behaviors of other drivers, such as lane changes, as well as their future trajectories given a recent window of past sensor observations. We address the first issue of semantic behavior prediction in this paper, which is a precursor to trajectory prediction, by introducing a framework that leverages the power of recurrent neural networks (RNNs) and graphical models. Our goal is to…
National Institute of Informatics（国立情报学研究所）；Carnegie Mellon University（卡内基梅隆大学）；General Motors Company（通用汽车公司）
Connected and Automated Vehicles (CAVs) have real-time information from the surrounding environment by using local on-board sensors, V2X (Vehicle-to-Everything) communications, pre-loaded vehicle-specific lookup tables, and map database. CAVs are capable of improving energy efficiency by incorporating these information. In particular, Eco-Cruise and Eco-Lane Selection on highways and/or motorways have immense potential to save energy, because there are generally fewer traffic controllers and the vehicles keep moving in general. In this paper, we present a cooperative and energy-efficient lane-…
Institute of Power Machinery and Vehicular Engineering, College of Energy Engineering, Zhejiang University（浙江大学能源工程学院动力机械及车辆工程研究所）
An automated driving system should have the ability to supervise its own performance and to request human driver to take over when necessary. In the lane keeping scenario, the prediction of vehicle future trajectory is the key to realize safe and trustworthy driving automation. Previous studies on vehicle trajectory prediction mainly fall into two categories, i.e. physics-based and manoeuvre-based methods. Using a physics-based methodology, this paper proposes a lane departure prediction algorithm based on closed-loop vehicle dynamics model. We use extended Kalman filter to estimate the curren…
Unmanned vehicle technologies are an area of great interest in theory and practice today. These technologies have advanced considerably after the first applications have been implemented and cause a rapid change in human life. Autonomous vehicles are also a big part of these technologies. The most important action of a driver has to do is to follow the lanes on the way to the destination. By using image processing and artificial intelligence techniques, an autonomous vehicle can move successfully without a driver help. They can go from the initial point to the specified target by applying pre-…
This paper presents a noval method that generates optimal trajectories for autonomous vehicles for in-lane driving scenarios. The method computes a trajectory using a two-phase optimization procedure. In the first phase, the optimization procedure generates a close-form driving guide line with differetiable curvatures. In the second phase, the procedure takes the driving guide line as input, and outputs dynamically feasible, jerk and time optimal trajectories for vehicles driving along the guide line. This method is especially useful for generating trajectories at curvy road where the vehicles…
LMT-Net: Lane Model Transformer Network for Automated HD Mapping from Sparse Vehicle Observations
Mercedes-Benz AG（梅赛德斯-奔驰股份公司）；Mercedes-Benz Research & Development North America（梅赛德斯-奔驰北美研发中心）；University of Stuttgart（斯图加特大学）；University of Southampton（南安普顿大学）
Developing, Analyzing, and Evaluating Vehicular Lane Keeping Algorithms Under Dynamic Lighting and Weather Conditions Using Electric Vehicles
Rice University（莱斯大学）；Pennsylvania State University（宾夕法尼亚州立大学）；Willamette University（威拉姆特大学）；Louisiana State University（路易斯安那州立大学）；Michigan State University（密歇根州立大学）；Lawrence Technological University（劳伦斯理工大学）
Machine Learning-Based Vehicle Intention Trajectory Recognition and Prediction for Autonomous Driving
University of Southern California（南加州大学）；Virginia Tech（弗吉尼亚理工大学）；Miami University（迈阿密大学）；Northern Arizona University（北亚利桑那大学）；Northeastern University（东北大学）
Understanding human driving behavior is crucial to develop autonomous vehicles' algorithms. However, most low level automation, such as the one in advanced driving assistance systems (ADAS), is based on objective safety measures, which are not always aligned with what the drivers perceive as safe and their correspondent driving behavior. Finding the bridge between the subjective perception and objective safety measures has been analyzed in this paper focusing specifically on lane-change scenarios. Results showed statistically significant differences between what is perceived as safe by drivers…
Johns Hopkins University（约翰斯·霍普金斯大学）；Samsung Strategy and Innovation Center, Samsung, Inc.（三星战略与创新中心，三星公司）
Trajectory prediction is crucial for autonomous vehicles. The planning system not only needs to know the current state of the surrounding objects but also their possible states in the future. As for vehicles, their trajectories are significantly influenced by the lane geometry and how to effectively use the lane information is of active interest. Most of the existing works use rasterized maps to explore road information, which does not distinguish different lanes. In this paper, we propose a novel instance-aware representation for lane representation. By integrating the lane features and traje…
Evaluating Low-Resource Lane Following Algorithms for Compute-Constrained Automated Vehicles
University of Minnesota（明尼苏达大学）；The University of Arizona（亚利桑那大学）；Old Dominion University（奥多明尼昂大学）；The Ohio State University（俄亥俄州立大学）；Butler University（巴特勒大学）；North Carolina A&T State University（北卡罗来纳农工州立大学）；New Mexico State University（新墨西哥州立大学）；Lawrence Technological University（劳伦斯理工大学）；Michigan State University（密歇根州立大学）
Department of Automation, Shanghai Jiao Tong University（上海交通大学自动化系）；Key Laboratory of System Control and Information Processing, Ministry of Education of China（中国教育部系统控制与信息处理重点实验室）；School of Electronic Information Engineering, Shanghai Dianji University（上海电机学院电子信息工程学院）；James Watt School of Engineering, University of Glasgow（格拉斯哥大学詹姆斯·瓦特工程学院）；School of Electronics and Information Engineering, Soochow University（苏州大学电子信息工程学院）
Lane change for autonomous vehicles (AVs) is an important but challenging task in complex dynamic traffic environments. Due to difficulties in guarantee safety as well as a high efficiency, AVs are inclined to choose relatively conservative strategies for lane change. To avoid the conservatism, this paper presents a cooperation-aware lane change method utilizing interactions between vehicles. We first propose an interactive trajectory prediction method to explore possible cooperations between an AV and the others. Further, an evaluation is designed to make a decision on lane change, in which s…
A robust estimation of road course and traffic lanes is an essential part of environment perception for next generations of Advanced Driver Assistance Systems and development of self-driving vehicles. In this paper, a flexible method for modeling multiple lanes in a vehicle in real time is presented. Information about traffic lanes, derived by cameras and other environmental sensors, that is represented as features, serves as input for an iterative expectation-maximization method to estimate a lane model. The generic and modular concept of the approach allows to freely choose the mathematical …
Autonomous driving promises to transform road transport. Multi-vehicle and multi-lane scenarios, however, present unique challenges due to constrained navigation and unpredictable vehicle interactions. Learning-based methods---such as deep reinforcement learning---are emerging as a promising approach to automatically design intelligent driving policies that can cope with these challenges. Yet, the process of safely learning multi-vehicle driving behaviours is hard: while collisions---and their near-avoidance---are essential to the learning process, directly executing immature policies on auton…
Key Laboratory of Road and Traffic Engineering of Ministry of Education, Tongji University（同济大学道路与交通工程教育部重点实验室）；China Railway Siyuan Survey and Design Group Co., Ltd.（中铁第四勘察设计院集团有限公司）；Institute of Logistics Science and Engineering, Shanghai Maritime University（上海海事大学物流科学与工程研究院）
The freeway on-ramp merging section is often identified as a crash-prone spot due to the high frequency of traffic conflicts. Very few traffic conflict analysis studies comprehensively consider different vehicle types at freeway merging section. Thus, the main objective of this study is to analyse conflicts between different vehicle types at freeway merging section. Field data are collected by Unmanned Aerial Vehicle (UAV) at merging areas in Shanghai, China. Vehicle extraction method is utilized to obtain vehicle trajectories. Time-to-collision (TTC) is utilized as the surrogate safety measur…
York University（约克大学）；University of Alcalá（阿尔卡拉大学）；European Commission - Joint Research Center（欧盟委员会联合研究中心）
In highway scenarios, an alert human driver will typically anticipate early cut-in/cut-out maneuvers of surrounding vehicles using visual cues mainly. Autonomous vehicles must anticipate these situations at an early stage too, to increase their safety and efficiency. In this work, lane-change recognition and prediction tasks are posed as video action recognition problems. Up to four different two-stream-based approaches, that have been successfully applied to address human action recognition, are adapted here by stacking visual cues from forward-looking video cameras to recognize and anticipat…
ARETE: Attention-based Rasterized Encoding for Topology Estimation using HSV-transformed Crowdsourced Vehicle Fleet Data
Mercedes-Benz AG, Research \& Development, Sindelfingen, Germany Institute for Intelligent Systems, Esslingen University of Applied Sciences, Esslingen, Germany
Georgia Institute of Technology（佐治亚理工学院）；Aison Pte. Ltd.（Aison私人有限公司）
Lane detection is an integral part of control systems in autonomous vehicles and lane departure warning systems as lanes are a key component of the operating environment for road vehicles. In a previous paper, a robust neural network output enhancement for active lane detection (RONELD) method augmenting deep learning lane detection models to improve active, or ego, lane accuracy performance was presented. This paper extends the work by further investigating the lane tracking methods used to increase robustness of the method to lane changes and different lane dimensions (e.g. lane marking thic…
Modeling the Lane-Change Reactions to Merging Vehicles for Highway On-Ramp Simulations
Global Center for Automotive Performance Simulation（全球汽车性能模拟中心）；Honda Research Institute, USA Inc.（美国本田研究所）；Virginia Tech Transportation Institute（弗吉尼亚理工大学交通研究所）
An extensive, precise and robust recognition and modeling of the environment is a key factor for next generations of Advanced Driver Assistance Systems and development of autonomous vehicles. In this paper, a real-time approach for the perception of multiple lanes on highways is proposed. Lane markings detected by camera systems and observations of other traffic participants provide the input data for the algorithm. The information is accumulated and fused using GraphSLAM and the result constitutes the basis for a multilane clothoid model. To allow incorporation of additional information sourc…
Northwestern University（西北大学）；University of California, Irvine（加利福尼亚大学欧文分校）
In the development of advanced driver-assistance systems (ADAS) and autonomous vehicles, machine learning techniques that are based on deep neural networks (DNNs) have been widely used for vehicle perception. These techniques offer significant improvement on average perception accuracy over traditional methods, however, have been shown to be susceptible to adversarial attacks, where small perturbations in the input may cause significant errors in the perception results and lead to system failure. Most prior works addressing such adversarial attacks focus only on the sensing and perception modu…
Institute of Industrial Science（产业技术研究所）；The University of Tokyo（东京大学）
In this article, the authors present a novel method to learn the personalized tactic of discretionary lane-change initiation for fully autonomous vehicles through human-computer interactions. Instead of learning from human-driving demonstrations, a reinforcement learning technique is employed to learn how to initiate lane changes from traffic context, the action of a self-driving vehicle, and in-vehicle user feedback. The proposed offline algorithm rewards the action-selection strategy when the user gives positive feedback and penalizes it when negative feedback. Also, a multi-dimensional driv…
New York University Tandon School of Engineering（纽约大学坦登工程学院）；C2SMART University Transportation Center（C2SMART大学交通中心）
Emergency vehicle (EMV) service is a key function of cities and is exceedingly challenging due to urban traffic congestion. A main reason behind EMV service delay is the lack of communication and cooperation between vehicles blocking EMVs. In this paper, we study the improvement of EMV service under V2I connectivity. We consider the establishment of dynamic queue jump lanes (DQJLs) based on real-time coordination of connected vehicles. We develop a novel Markov decision process formulation for the DQJL problem, which explicitly accounts for the uncertainty of drivers' reaction to approaching E…
University of Massachusetts（马萨诸塞大学）；Nissan Research Center（日产研究中心）
Autonomous vehicles (AVs) require accurate metric and topological location estimates for safe, effective navigation and decision-making. Although many high-definition (HD) roadmaps exist, they are not always accurate since public roads are dynamic, shaped unpredictably by both human activity and nature. Thus, AVs must be able to handle situations in which the topology specified by the map does not agree with reality. We present the Variable Structure Multiple Hidden Markov Model (VSM-HMM) as a framework for localizing in the presence of topological uncertainty, and demonstrate its effectivenes…
Neural network-based driving planners have shown great promises in improving task performance of autonomous driving. However, it is critical and yet very challenging to ensure the safety of systems with neural network based components, especially in dense and highly interactive traffic environments. In this work, we propose a safety-driven interactive planning framework for neural network-based lane changing. To prevent over conservative planning, we identify the driving behavior of surrounding vehicles and assess their aggressiveness, and then adapt the planned trajectory for the ego vehicle …
In this paper, we study how to learn an appropriate lane changing strategy for autonomous vehicles by using deep reinforcement learning. We show that the reward of the system should consider the overall traffic efficiency instead of the travel efficiency of an individual vehicle. In summary, cooperation leads to a more harmonic and efficient traffic system rather than competition
National Central University（国立中央大学）；Tokyo Metropolitan University（东京都立大学）；Auburn University（奥本大学）
Reasoning about vehicle path prediction is an essential and challenging problem for the safe operation of autonomous driving systems. There exist many research works for path prediction. However, most of them do not use lane information and are not based on the Transformer architecture. By utilizing different types of data collected from sensors equipped on the self-driving vehicles, we propose a path prediction system named Multi-modal Transformer Path Prediction (MTPP) that aims to predict long-term future trajectory of target agents. To achieve more accurate path prediction, the Transformer…
Graz University of Technology（格拉茨技术大学）；MAGNA Steyr Fahrzeugtechnik AG & CO KG（麦格纳斯太尔汽车技术股份两合公司）
Virtual development and prototyping has already become an integral part in the field of automated driving systems (ADS). There are plenty of software tools that are used for the virtual development of ADS. One such tool is CarMaker from IPG Automotive, which is widely used in the scientific community and in the automotive industry. It offers a broad spectrum of implementation and modelling possibilities of the vehicle, driver behavior, control, sensors, and environmental models. Focusing on the virtual development of highly automated driving functions on the vehicle guidance level, it is essen…
Multi-lane roads are typical scenarios in the real-world traffic system. Vehicles usually have preference on lanes according to their routes and destinations. Few of the existing studies looks into the problem of controlling vehicles to drive on their desired lanes. This paper proposes a formation control method that considers vehicles' preference on different lanes. The bi-level formation control framework is utilized to plan collision-free motion for vehicles, where relative target assignment and path planning are performed in the upper level, and trajectory planning and tracking are perform…
We introduce a prioritized system-optimal algorithm for mandatory lane change (MLC) behavior of connected and automated vehicles (CAV) from a dedicated lane. Our approach applies a cooperative lane change that prioritizes the decisions of lane changing vehicles which are closer to the end of the diverging zone (DZ), and optimizes the predicted total system travel time. Our experiments on synthetic data show that the proposed algorithm improves the traffic network efficiency by attaining higher speeds in the dedicated lane and earlier MLC positions while ensuring a low computational time. Our a…
Universidad de Alcala（阿尔卡拉大学）；Università degli Studi di Milano-Bicocca（米兰比可卡大学）；University of Freiburg（弗莱堡大学）；Nanyang Technological University（南洋理工大学）
We present a probabilistic ego-lane estimation algorithm for highway-like scenarios that is designed to increase the accuracy of the ego-lane estimate, which can be obtained relying only on a noisy line detector and tracker. The contribution relies on a Hidden Markov Model (HMM) with a transient failure model. The proposed algorithm exploits the OpenStreetMap (or other cartographic services) road property lane number as the expected number of lanes and leverages consecutive, possibly incomplete, observations. The algorithm effectiveness is proven by employing different line detectors and showi…
Autonomy and connectivity are considered among the most promising technologies to improve safety, mobility, fuel and time consumption in transportation systems. Some of the fuel efficiency benefits of connected and automated vehicles (CAVs) can be realized through platooning. A platoon is a virtual train of CAVs that travel together following the platoon head, with small gaps between them. Vehicles may also reduce travel time by lane changing. In this paper, we devise an optimal control-based trajectory planning model that can provide safe and efficient trajectories for the subject vehicle and…
Tongji University（同济大学）；University of Massachusetts Amherst（马萨诸塞大学阿默斯特分校）
Lane-changing (LC) behavior describes the lateral movement of the vehicle from the current-lane to the target-lane while proceeding forward. Among the many research directions, LC duration (LCD) measures the total time it takes for a vehicle to travel from the current lane to the target lane, which is an indispensable indicator to characterize the LC behavior. Although existing research has made some achievements, less attention has been paid to the research of heavy vehicles' LCD. Therefore, this paper aims to further explore the LCD between heavy vehicles and passenger cars. LC trajectories …
When the traffic stream is extremely congested and surrounding vehicles are not cooperative, the mandatory lane changing can be significantly difficult. In this work, we propose an interactive trajectory planner, which will firstly attempt to change lanes as long as safety is ensured. Based on receding horizon planning, the ego vehicle can abort or continue changing lanes according to surrounding vehicles' reactions. We demonstrate the performance of our planner in extensive simulations with eight surrounding vehicles, initial velocity ranging from 0.5 to 5 meters per second, and bumper to bum…
This paper introduces Sentinel, an onboard system for intelligent vehicles that guides their lane changing behavior during a freeway incident with the goal of reducing traffic congestion, capacity drop, and delay. When an incident blocking the lanes ahead is detected, Sentinel calculates the probability of leaving the blocked lane(s) before reaching the incident point at each time step. It advises the vehicle to leave the blocked lane(s) when that probability drops below a certain threshold, as the vehicle nears the congestion boundary. By doing this, Sentinel reduces the number of late-stage …
Lane change is a crucial vehicle maneuver which needs coordination with surrounding vehicles. Automated lane changing functions built on rule-based models may perform well under pre-defined operating conditions, but they may be prone to failure when unexpected situations are encountered. In our study, we proposed a Reinforcement Learning based approach to train the vehicle agent to learn an automated lane change behavior such that it can intelligently make a lane change under diverse and even unforeseen scenarios. Particularly, we treated both state space and action space as continuous, and de…
Islamic University of Technology (IUT)（伊斯兰科技大学（IUT））
This paper discusses a vehicle prototype that recognizes streets' lanes and plans its motion accordingly without any human input. Pi Camera 1.3 captures real-time video, which is then processed by Raspberry-Pi 3.0 Model B. The image processing algorithms are written in Python 3.7.4 with OpenCV 4.2. Arduino Uno is utilized to control the PID algorithm that controls the motor controller, which in turn controls the wheels. Algorithms that are used to detect the lanes are the Canny edge detection algorithm and Hough transformation. Elementary algebra is used to draw the detected lanes. After detec…
University of Alcalá（阿尔卡拉大学）；York University（约克大学）
In highway scenarios, an alert human driver will typically anticipate early cut-in and cut-out maneuvers of surrounding vehicles using only visual cues. An automated system must anticipate these situations at an early stage too, to increase the safety and the efficiency of its performance. To deal with lane-change recognition and prediction of surrounding vehicles, we pose the problem as an action recognition/prediction problem by stacking visual cues from video cameras. Two video action recognition approaches are analyzed: two-stream convolutional networks and spatiotemporal multiplier networ…
Mercedes-Benz AG（梅赛德斯-奔驰股份公司）；Ulm University（乌尔姆大学）；RWTH Aachen University（亚琛工业大学）；University of Siegen（锡根大学）
To plan safe and comfortable trajectories for automated vehicles on highways, accurate predictions of traffic situations are needed. So far, a lot of research effort has been spent on detecting lane change maneuvers rather than on estimating the point in time a lane change actually happens. In practice, however, this temporal information might be even more useful. This paper deals with the development of a system that accurately predicts the time to the next lane change of surrounding vehicles on highways using long short-term memory-based recurrent neural networks. An extensive evaluation bas…
As the advanced driver assistance system (ADAS) functions become more sophisticated, the strategies that properly coordinate interaction and communication among the ADAS functions are required for autonomous driving. This paper proposes a derivative-free optimization based imitation learning method for the decision maker that coordinates the proper ADAS functions. The proposed method is able to make decisions in multi-lane highways timely with the LIDAR data. The simulation-based evaluation verifies that the proposed method presents desired performance.
