Humans make daily routine decisions based on their internal states in intricate interaction scenarios. This paper presents a probabilistically reconstructive learning approach to identify the internal states of multi-vehicle sequential interactions when merging at highway on-ramps. We treated the merging task's sequential decision as a dynamic, stochastic process and then integrated the internal states into an HMM-GMR model, a probabilistic combination of an extended Gaussian mixture regression (GMR) and hidden Markov models (HMM). We also developed a variant expectation-maximum (EM) algorithm to estimate the model parameters and verified it based on a real-world data set. Experiment results reveal that three interpretable internal states can semantically describe the interactive merge procedure at highway on-ramps. This finding provides a basis to develop an efficient model-based decision-making algorithm for autonomous vehicles (AVs) in a partially observable environment.
Beijing Institute of Technology（北京理工大学）；University of California at Berkeley（加州大学伯克利分校）；McGill University（麦吉尔大学）
Merging at highway on-ramps while interacting with other human-driven vehicles is challenging for autonomous vehicles (AVs). An efficient route to this challenge requires exploring and exploiting knowledge of the interaction process from demonstrations by humans. However, it is unclear what information (or environmental states) is utilized by the human driver to guide their behavior throughout the whole merging process. This paper provides quantitative analysis and evaluation of the merging behavior at highway on-ramps with congested traffic in a volume of time and space. Two types of social i…
Modeling the Lane-Change Reactions to Merging Vehicles for Highway On-Ramp Simulations
Global Center for Automotive Performance Simulation（全球汽车性能模拟中心）；Honda Research Institute, USA Inc.（美国本田研究所）；Virginia Tech Transportation Institute（弗吉尼亚理工大学交通研究所）
From Observation to Prediction: LSTM for Vehicle Lane Change Forecasting on Highway On/Off-Ramps
C-DRiVeS Lab: Cognitive Driving Research in Vehicular Systems（C-DRiVeS实验室：车载系统认知驾驶研究）；Computer Science and Engineering Department - Faculty of Media Engineering and Technology - German University in Cairo（计算机科学与工程系 - 媒体工程与技术学院 - 埃及德国大学）
Global Center for Automotive Performance Simulation（全球汽车性能模拟中心）；Honda Research Institute, USA Inc.（美国本田研究院）；Virginia Tech Transportation Institute（弗吉尼亚理工交通研究所）
Delay-Aware Reinforcement Learning for Highway On-Ramp Merging under Stochastic Communication Latency
Department of Computer Science, George Washington University, Washington, D.C.（计算机科学系，乔治华盛顿大学，华盛顿特区）；Connected and Automated Vehicle Program Manager, Traffic Operations Division, Virginia Department of Transportation（连接与自动化车辆计划主任，交通运营处，弗吉尼亚州交通部）；Department of Mechanical & Aerospace Engineering, George Washington University, Washington, D.C.（机械与航空航天工程系，乔治华盛顿大学，华盛顿特区）
Shanghai University（上海大学）；Michigan State University（密歇根州立大学）
Deep reinforcement learning (DRL) has a great potential for solving complex decision-making problems in autonomous driving, especially in mixed-traffic scenarios where autonomous vehicles and human-driven vehicles (HDVs) drive together. Safety is a key during both the learning and deploying reinforcement learning (RL) algorithms process. In this paper, we formulate the on-ramp merging as a Markov Decision Process (MDP) problem and solve it with an off-policy RL algorithm, i.e., Soft Actor-Critic for Discrete Action Settings (SAC-Discrete). In addition, a motion predictive safety controller inc…
Alibaba-NTU Singapore Joint Research Institute, Nanyang Technological University（南洋理工大学阿里巴巴-南洋理工大学新加坡联合研究院）；DAMO Academy, Alibaba Group（阿里巴巴集团达摩院）；Shanghai Jiao Tong University（上海交通大学）；Nanyang Technological University（南洋理工大学）
It has become a consensus that autonomous vehicles (AVs) will first be widely deployed on highways. However, the complexity of highway interchanges becomes the bottleneck for deploying AVs. An AV should be sufficiently tested under different highway interchanges, which is still challenging due to the lack of available datasets containing diverse highway interchanges. In this paper, we propose a model-driven method, FLYOVER, to generate a dataset consisting of diverse interchanges with measurable diversity coverage. First, FLYOVER proposes a labeled digraph to model the topology of an interchan…
School of Vehicle and Mobility, Tsinghua University（清华大学车辆与学院）
Merging into the highway from the on-ramp is an essential scenario for automated driving. The decision-making under the scenario needs to balance the safety and efficiency performance to optimize a long-term objective, which is challenging due to the dynamic, stochastic, and adversarial characteristics. The Rule-based methods often lead to conservative driving on this task while the learning-based methods have difficulties meeting the safety requirements. In this paper, we propose an RL-based end-to-end decision-making method under a framework of offline training and online correction, called …
Automated Digital Twin Construction for Highway Scenarios Using LiDAR Point Clouds and OpenStreetMap
Institute of Automotive Engineering, Graz University of Technology（格拉茨技术大学汽车工程研究所）；School of Intelligent Connected Vehicle, Hubei University of Automotive Technology（湖北汽车工业学院智能网联汽车学院）；Professorship of Autonomous Vehicle Systems, Technical University of Munich（慕尼黑工业大学自动驾驶系统教席）
Reinforcement Learning Based Oscillation Dampening: Scaling up Single-Agent RL algorithms to a 100 AV highway field operational test
University of California, Berkeley（加州大学伯克利分校）；École des Ponts Paristech（巴黎高科路桥学院）；Vanderbilt University（范德堡大学）；Temple University（天普大学）；Rutgers University-Camden（罗格斯大学卡姆登分校）
Fault-Tolerant MARL for CAVs under Observation Perturbations for Highway On-Ramp Merging
Department of Automation, Tsinghua University（自动化系，清华大学）；Beijing National Research Center for Information Science and Technology (BNRist), Tsinghua University（北京信息科学与技术国家研究中心（BNRist），清华大学）
Safe Reinforcement Learning of Autonomous Highway Driving: A Unified Framework for Safety and Efficiency
School of Physics, Northeast Normal University（东北师范大学物理学院）；Clean Energy Automotive Engineering Center, School of Automotive Studies, Tongji University（同济大学汽车学院清洁能源汽车工程中心）；Mengshi Automobile Technology Company, Dongfeng Motor Corporation（东风汽车公司猛士汽车技术公司）
POMDP-Based Trajectory Planning for On-Ramp Highway Merging
Faculty of Electrical Engineering, Czech Technical University in Prague（布拉格捷克技术大学电气工程学院）；Czech Institute of Informatics, Robotics and Cybernetics, Czech Technical University in Prague（布拉格捷克技术大学捷克信息学、机器人学与控制论研究所）
German Aerospace Center (DLR)（德国航空航天中心）；Hochschule Emden/Leer, University of Applied Sciences（埃姆登/莱尔应用科学大学）
Connected and Automated Vehicles (CAVs) are envisioned to transform the future industrial and private transportation sectors. However, due to the system's enormous complexity, functional verification and validation of safety aspects are essential before the technology merges into the public domain. Therefore, in recent years, a scenario-driven approach has gained acceptance, emphasizing the requirement of a solid data basis of scenarios. The large-scale research facility Test Bed Lower Saxony (TFNDS) enables the provision of ample information for a database of scenarios on highways. For that p…
MR-LDM -- The Merge-Reactive Longitudinal Decision Model: Game Theoretic Human Decision Modeling for Interactive Sim Agents
Global Center for Automotive Performance Simulation（全球汽车性能模拟中心）；Honda Research Institute, USA Inc.（本田研究院（美国公司））；Division of Data and Analytics, Virginia Tech Transportation Institute（弗吉尼亚理工运输研究所数据分析部门）
Ramp merging is a typical application of cooperative intelligent transportation system (C-ITS). Vehicle trajectories perceived by roadside sensors are importation complement to the limited visual field of on-board perception. Vehicle tracking and trajectory denoising algorithm is proposed in this paper to take full advantage of roadside cameras for vehicle trajectory and speed profile estimation. Dynamic speed guidance algorithm is proposed to help on-ramp vehicles to merge into mainline smoothly, even in non-cooperative environment where mainline vehicles are not expected to slow down to acco…
Prior work has looked at applying reinforcement learning and imitation learning approaches to autonomous driving scenarios, but either the safety or the efficiency of the algorithm is compromised. With the use of control barrier functions embedded into the reinforcement learning policy, we arrive at safe policies to optimize the performance of the autonomous driving vehicle. However, control barrier functions need a good approximation of the model of the car. We use probabilistic control barrier functions as an estimate of the model uncertainty. The algorithm is implemented as an online versio…
Autonomous vehicles should be able to generate accurate probabilistic predictions for uncertain behavior of other road users. Moreover, reactive predictions are necessary in highly interactive driving scenarios to answer "what if I take this action in the future" for autonomous vehicles. There is no existing unified framework to homogenize the problem formulation, representation simplification, and evaluation metric for various prediction methods, such as probabilistic graphical models (PGM), neural networks (NN) and inverse reinforcement learning (IRL). In this paper, we formulate a probabili…
University of Illinois at Urbana-Champaign（伊利诺伊大学厄巴纳-香槟分校）；University of Texas at Austin（得克萨斯大学奥斯汀分校）
We consider the problem of designing an algorithm to allow a car to autonomously merge on to a highway from an on-ramp. Two broad classes of techniques have been proposed to solve motion planning problems in autonomous driving: Model Predictive Control (MPC) and Reinforcement Learning (RL). In this paper, we first establish the strengths and weaknesses of state-of-the-art MPC and RL-based techniques through simulations. We show that the performance of the RL agent is worse than that of the MPC solution from the perspective of safety and robustness to out-of-distribution traffic patterns, i.e.,…
Beijing Institute of Technology（北京理工大学）；Delft University of Technology（代尔夫特理工大学）；Nanyang Technological University（南洋理工大学）
An efficient and reliable multi-agent decision-making system is highly demanded for the safe and efficient operation of connected autonomous vehicles in intelligent transportation systems. Current researches mainly focus on the Deep Reinforcement Learning (DRL) methods. However, utilizing DRL methods in interactive traffic scenarios is hard to represent the mutual effects between different vehicles and model the dynamic traffic environments due to the lack of interactive information in the representation of the environments, which results in low accuracy of cooperative decisions generation. To…
Beijing University of Technology（北京工业大学）；McGill University（麦吉尔大学）
Humans are experts in making decisions for challenging driving tasks with uncertainties. Many efforts have been made to model the decision-making process of human drivers at the behavior level. However, limited studies explain how human drivers actively make reliable sequential decisions to complete interactive driving tasks in an uncertain environment. This paper argues that human drivers intently search for actions to reduce the uncertainty of their perception of the environment, i.e., perceptual uncertainty, to a low level that allows them to make a trustworthy decision easily. This paper p…
Towards A General-Purpose Motion Planning for Autonomous Vehicles Using Fluid Dynamics
WMG, University of Warwick（华威大学WMG学院）；School of Electronics, Electrical Engineering and Computer Science, Queen’s University of Belfast（贝尔法斯特女王大学电子、电气工程与计算机科学学院）
University of California, Berkeley（加州大学伯克利分校）；Xi’an Jiaotong University（西安交通大学）；Harbin Institute of Technology（哈尔滨工业大学）；MINES ParisTech（巴黎高等矿业学院）；FZI Research Center for Information Technology（FZI信息技术研究中心）；Karlsruhe Institute of Technology（卡尔斯鲁厄理工学院）
Behavior-related research areas such as motion prediction/planning, representation/imitation learning, behavior modeling/generation, and algorithm testing, require support from high-quality motion datasets containing interactive driving scenarios with different driving cultures. In this paper, we present an INTERnational, Adversarial and Cooperative moTION dataset (INTERACTION dataset) in interactive driving scenarios with semantic maps. Five features of the dataset are highlighted. 1) The interactive driving scenarios are diverse, including urban/highway/ramp merging and lane changes, roundab…
