Merging into the highway from the on-ramp is an essential scenario for automated driving. The decision-making under the scenario needs to balance the safety and efficiency performance to optimize a long-term objective, which is challenging due to the dynamic, stochastic, and adversarial characteristics. The Rule-based methods often lead to conservative driving on this task while the learning-based methods have difficulties meeting the safety requirements. In this paper, we propose an RL-based end-to-end decision-making method under a framework of offline training and online correction, called the Shielded Distributional Soft Actor-critic (SDSAC). The SDSAC adopts the policy evaluation with safety consideration and a safety shield parameterized with the barrier function in its offline training and online correction, respectively. These two measures support each other for better safety while not damaging the efficiency performance severely. We verify the SDSAC on an on-ramp merge scenario in simulation. The results show that the SDSAC has the best safety performance compared to baseline algorithms and achieves efficient driving simultaneously.
Ramp merging is a critical maneuver for road safety and traffic efficiency. Most of the current automated driving systems developed by multiple automobile manufacturers and suppliers are typically limited to restricted access freeways only. Extending the automated mode to ramp merging zones presents substantial challenges. One is that the automated vehicle needs to incorporate a future objective (e.g. a successful and smooth merge) and optimize a long-term reward that is impacted by subsequent actions when executing the current action. Furthermore, the merging process involves interaction betw…
Michigan Technological University（密歇根理工大学）；University of Science and Technology of China（中国科学技术大学）
Despite the success of AI-enabled onboard perception, on-ramp merging has been one of the main challenges for autonomous driving. Due to limited sensing range of onboard sensors, a merging vehicle can hardly observe main road conditions and merge properly. By leveraging the wireless communications between connected and automated vehicles (CAVs), a merging CAV has potential to proactively obtain the intentions of nearby vehicles. However, CAVs can be prone to inaccurate observations, such as the noisy basic safety messages (BSM) and poor quality surveillance images. In this paper, we present a …
California PATH, University of California, Berkeley（加州大学伯克利分校加州PATH）
Multiple automakers have in development or in production automated driving systems (ADS) that offer freeway-pilot functions. This type of ADS is typically limited to restricted-access freeways only, that is, the transition from manual to automated modes takes place only after the ramp merging process is completed manually. One major challenge to extend the automation to ramp merging is that the automated vehicle needs to incorporate and optimize long-term objectives (e.g. successful and smooth merge) when near-term actions must be safely executed. Moreover, the merging process involves interac…
Modeling the Lane-Change Reactions to Merging Vehicles for Highway On-Ramp Simulations
Global Center for Automotive Performance Simulation（全球汽车性能模拟中心）；Honda Research Institute, USA Inc.（美国本田研究所）；Virginia Tech Transportation Institute（弗吉尼亚理工大学交通研究所）
Global Center for Automotive Performance Simulation（全球汽车性能模拟中心）；Honda Research Institute, USA Inc.（美国本田研究院）；Virginia Tech Transportation Institute（弗吉尼亚理工交通研究所）
University of Illinois at Urbana-Champaign（伊利诺伊大学厄巴纳-香槟分校）；University of Texas at Austin（得克萨斯大学奥斯汀分校）
We consider the problem of designing an algorithm to allow a car to autonomously merge on to a highway from an on-ramp. Two broad classes of techniques have been proposed to solve motion planning problems in autonomous driving: Model Predictive Control (MPC) and Reinforcement Learning (RL). In this paper, we first establish the strengths and weaknesses of state-of-the-art MPC and RL-based techniques through simulations. We show that the performance of the RL agent is worse than that of the MPC solution from the perspective of safety and robustness to out-of-distribution traffic patterns, i.e.,…
School of Mechanical Engineering, Beijing Institute of Technology（北京理工大学机械工程学院）；University of California at Berkeley（加州大学伯克利分校）
Humans make daily routine decisions based on their internal states in intricate interaction scenarios. This paper presents a probabilistically reconstructive learning approach to identify the internal states of multi-vehicle sequential interactions when merging at highway on-ramps. We treated the merging task's sequential decision as a dynamic, stochastic process and then integrated the internal states into an HMM-GMR model, a probabilistic combination of an extended Gaussian mixture regression (GMR) and hidden Markov models (HMM). We also developed a variant expectation-maximum (EM) algorithm…
Honda Research Institute (HRI) Europe（本田欧洲研究所）；Honda Innovation Lab (HIL) Tokyo, Honda R&D Co., Ltd.（本田技研株式会社东京本田创新实验室）
We consider the problem of correct motion planning for T-intersection merge-ins of arbitrary geometry and vehicle density. A merge-in support system has to estimate the chances that a gap between two consecutive vehicles can be taken successfully. In contrast to previous models based on heuristic gap size rules, we present an approach which optimizes the integral risk of the situation using parametrized velocity ramps. It accounts for the risks from curves and all involved vehicles (front and rear on all paths) with a so-called survival analysis. For comparison, we also introduce a specially d…
Fault-Tolerant MARL for CAVs under Observation Perturbations for Highway On-Ramp Merging
Department of Automation, Tsinghua University（自动化系，清华大学）；Beijing National Research Center for Information Science and Technology (BNRist), Tsinghua University（北京信息科学与技术国家研究中心（BNRist），清华大学）
MR-LDM -- The Merge-Reactive Longitudinal Decision Model: Game Theoretic Human Decision Modeling for Interactive Sim Agents
Global Center for Automotive Performance Simulation（全球汽车性能模拟中心）；Honda Research Institute, USA Inc.（本田研究院（美国公司））；Division of Data and Analytics, Virginia Tech Transportation Institute（弗吉尼亚理工运输研究所数据分析部门）
PPO-Based Vehicle Control for Ramp Merging Scheme Assisted by Enhanced C-V2X
School of Internet of Things Engineering, Jiangnan University（物联网工程学院，江南大学）；Department of Electronic Engineering, State Key laboratory of Space Network and Communications, Beijing National Research Center for Information Science and Technology, Tsinghua University（电子工程系，空间网络与通信国家重点实验室，北京信息科学与技术国家研究中心，清华大学）；Department of Computer Science, Brunel University（计算机科学系，布鲁内尔大学）；State Key Lab. of ISN and School of Telecommunications Engineering, Xidian University（信息国家实验室和电信工程学院，西安电子科技大学）；Department of Electronic Engineering, Shanghai Jiao Tong University（电子工程系，上海交通大学）；Department of Electrical and Computer Engineering, the Hong Kong University of Science and Technology (HKUST)（电气与计算机工程系，香港科技大学）
Ramp merging is a typical application of cooperative intelligent transportation system (C-ITS). Vehicle trajectories perceived by roadside sensors are importation complement to the limited visual field of on-board perception. Vehicle tracking and trajectory denoising algorithm is proposed in this paper to take full advantage of roadside cameras for vehicle trajectory and speed profile estimation. Dynamic speed guidance algorithm is proposed to help on-ramp vehicles to merge into mainline smoothly, even in non-cooperative environment where mainline vehicles are not expected to slow down to acco…
Toyota Central R & D Labs., Inc.（丰田中央研发实验室有限公司）；Toyota R & D（丰田研发中心）；University of Georgia（佐治亚大学）
Freeway merging in congested traffic is a significant challenge toward fully automated driving. Merging vehicles need to decide not only how to merge into a spot, but also where to merge. We present a method for the freeway merging based on multi-policy decision making with a reinforcement learning method called {\em passive actor-critic} (pAC), which learns with less knowledge of the system and without active exploration. The method selects a merging spot candidate by using the state value learned with pAC. We evaluate our method using real traffic data. Our experiments show that pAC achieves…
Amazon（亚马逊）；University of Massachusetts Lowell（马萨诸塞大学洛厄尔分校）
Given the aging infrastructure and the anticipated growing number of highway work zones in the United States, it is important to investigate work zone merge control, which is critical for improving work zone safety and capacity. This paper proposes and evaluates a novel highway work zone merge control strategy based on cooperative driving behavior enabled by artificial intelligence. The proposed method assumes that all vehicles are fully automated, connected and cooperative. It inserts two metering zones in the open lane to make space for merging vehicles in the closed lane. In addition, each …
German Aerospace Center (DLR)（德国航空航天中心）；Hochschule Emden/Leer, University of Applied Sciences（埃姆登/莱尔应用科学大学）
Connected and Automated Vehicles (CAVs) are envisioned to transform the future industrial and private transportation sectors. However, due to the system's enormous complexity, functional verification and validation of safety aspects are essential before the technology merges into the public domain. Therefore, in recent years, a scenario-driven approach has gained acceptance, emphasizing the requirement of a solid data basis of scenarios. The large-scale research facility Test Bed Lower Saxony (TFNDS) enables the provision of ample information for a database of scenarios on highways. For that p…
Designing an LLM-Based Copilot for Manufacturing Equipment Selection
RWTH Aachen University（亚琛工业大学）；Hong Kong Industrial Artificial Intelligence & Robotics Centre (FLAIR)（香港工业人工智能与机器人中心（FLAIR））；INC Innovation Center GmbH（INC创新中心有限公司）
