PPO-Based Vehicle Control for Ramp Merging Scheme Assisted by Enhanced C-V2X
School of Internet of Things Engineering, Jiangnan University（物联网工程学院，江南大学）；Department of Electronic Engineering, State Key laboratory of Space Network and Communications, Beijing National Research Center for Information Science and Technology, Tsinghua University（电子工程系，空间网络与通信国家重点实验室，北京信息科学与技术国家研究中心，清华大学）；Department of Computer Science, Brunel University（计算机科学系，布鲁内尔大学）；State Key Lab. of ISN and School of Telecommunications Engineering, Xidian University（信息国家实验室和电信工程学院，西安电子科技大学）；Department of Electronic Engineering, Shanghai Jiao Tong University（电子工程系，上海交通大学）；Department of Electrical and Computer Engineering, the Hong Kong University of Science and Technology (HKUST)（电气与计算机工程系，香港科技大学）
Qiong Wu, Maoxin Ji, Pingyi Fan, Kezhi Wang, Nan Cheng, Wen Chen, Khaled B. Letaief
匝道汇入是自动驾驶中的关键挑战，因为汇入车道的车辆需要动态调整位置和速度，同时监测主路交通以避免碰撞。为解决这一挑战，我们提出了一种基于强化学习的新型汇入控制方案，该方案整合了横向控制机制。该方法确保汇入车道的车辆平稳并入主路，同时优化燃油效率和乘客舒适度。此外，我们认识到车对车（V2V）通信对控制策略的影响，并引入了一种利用蜂窝车联网（Cellular Vehicle-to-Everything, C-V2X）Mode 4的增强协议。该协议旨在降低信息年龄（Age of Information, AoI）并提升通信可靠性。在仿真实验中，我们采用两种基于AoI的指标，严格评估该协议在自动驾驶场景下的有效性。我们将NS3网络仿真器与Python结合，同时仿真V2V通信与车辆控制。结果表明，增强型C-V2X Mode 4的性能优于标准版本，同时所提出的控制方案可确保车辆在匝道汇入过程中安全可靠地运行。
英文摘要：
On-ramp merging presents a critical challenge in autonomous driving, as vehicles from merging lanes need to dynamically adjust their positions and speeds while monitoring traffic on the main road to prevent collisions. To address this challenge, we propose a novel merging control scheme based on reinforcement learning, which integrates lateral control mechanisms. This approach ensures the smooth integration of vehicles from the merging lane onto the main road, optimizing both fuel efficiency and passenger comfort. Furthermore, we recognize the impact of vehicle-to-vehicle (V2V) communication on control strategies and introduce an enhanced protocol leveraging Cellular Vehicle-to-Everything (C-V2X) Mode 4. This protocol aims to reduce the Age of Information (AoI) and improve communication reliability. In our simulations, we employ two AoI-based metrics to rigorously assess the protocol's effectiveness in autonomous driving scenarios. By combining the NS3 network simulator with Python, we simulate V2V communication and vehicle control simultaneously. The results demonstrate that the enhanced C-V2X Mode 4 outperforms the standard version, while the proposed control scheme ensures safe and reliable vehicle operation during on-ramp merging.
Michigan Technological University（密歇根理工大学）；University of Science and Technology of China（中国科学技术大学）
Despite the success of AI-enabled onboard perception, on-ramp merging has been one of the main challenges for autonomous driving. Due to limited sensing range of onboard sensors, a merging vehicle can hardly observe main road conditions and merge properly. By leveraging the wireless communications between connected and automated vehicles (CAVs), a merging CAV has potential to proactively obtain the intentions of nearby vehicles. However, CAVs can be prone to inaccurate observations, such as the noisy basic safety messages (BSM) and poor quality surveillance images. In this paper, we present a …
Prior work has looked at applying reinforcement learning and imitation learning approaches to autonomous driving scenarios, but either the safety or the efficiency of the algorithm is compromised. With the use of control barrier functions embedded into the reinforcement learning policy, we arrive at safe policies to optimize the performance of the autonomous driving vehicle. However, control barrier functions need a good approximation of the model of the car. We use probabilistic control barrier functions as an estimate of the model uncertainty. The algorithm is implemented as an online versio…
University of Illinois at Urbana-Champaign（伊利诺伊大学厄巴纳-香槟分校）；University of Texas at Austin（得克萨斯大学奥斯汀分校）
We consider the problem of designing an algorithm to allow a car to autonomously merge on to a highway from an on-ramp. Two broad classes of techniques have been proposed to solve motion planning problems in autonomous driving: Model Predictive Control (MPC) and Reinforcement Learning (RL). In this paper, we first establish the strengths and weaknesses of state-of-the-art MPC and RL-based techniques through simulations. We show that the performance of the RL agent is worse than that of the MPC solution from the perspective of safety and robustness to out-of-distribution traffic patterns, i.e.,…
California PATH, University of California, Berkeley（加州大学伯克利分校加州PATH）
Multiple automakers have in development or in production automated driving systems (ADS) that offer freeway-pilot functions. This type of ADS is typically limited to restricted-access freeways only, that is, the transition from manual to automated modes takes place only after the ramp merging process is completed manually. One major challenge to extend the automation to ramp merging is that the automated vehicle needs to incorporate and optimize long-term objectives (e.g. successful and smooth merge) when near-term actions must be safely executed. Moreover, the merging process involves interac…
Fault-Tolerant MARL for CAVs under Observation Perturbations for Highway On-Ramp Merging
Department of Automation, Tsinghua University（自动化系，清华大学）；Beijing National Research Center for Information Science and Technology (BNRist), Tsinghua University（北京信息科学与技术国家研究中心（BNRist），清华大学）
A Cascading Cooperative Multi-agent Framework for On-ramp Merging Control Integrating Large Language Models
Shenzhen International Graduate School, Tsinghua University（清华大学深圳国际研究生院）；Department of Computer Science & Engineering, University of Minnesota（明尼苏达大学计算机科学与工程系）；Department of Mechanical Engineering & Materials Science, Yale University（耶鲁大学机械工程与材料科学系）；Computer Vision, BYD COMPANY LIMITED（计算机视觉，比亚迪有限公司）；School of Architecture, University of Texas at Austin（德克萨斯大学奥斯汀分校建筑学院）；Faculty of Applied Science & Engineering, University of Toronto（多伦多大学应用科学与工程学院）
Coordinated Lane-Level Variable Speed Limits and Ramp Metering for Successive Weaving Segments Considering Merging/Diverging Risks: A Hybrid Model Predictive Control and Multi-Agent Reinforcement Learning Approach
School of Vehicle and Mobility, Tsinghua University（清华大学车辆与学院）
Merging into the highway from the on-ramp is an essential scenario for automated driving. The decision-making under the scenario needs to balance the safety and efficiency performance to optimize a long-term objective, which is challenging due to the dynamic, stochastic, and adversarial characteristics. The Rule-based methods often lead to conservative driving on this task while the learning-based methods have difficulties meeting the safety requirements. In this paper, we propose an RL-based end-to-end decision-making method under a framework of offline training and online correction, called …
V2X-Assisted Distributed Computing and Control Framework for Connected and Automated Vehicles under Ramp Merging Scenario
School of Internet of Things Engineering, Jiangnan University（江南大学物联网工程学院）；Tsinghua University（清华大学）；Brunel University London（伦敦布鲁内尔大学）；Xidian University（西安电子科技大学）；Shanghai Jiao Tong University（上海交通大学）；the Hong Kong University of Science and Technology (HKUST)（香港科技大学）
Air traffic control is a real-time safety-critical decision making process in highly dynamic and stochastic environments. In today's aviation practice, a human air traffic controller monitors and directs many aircraft flying through its designated airspace sector. With the fast growing air traffic complexity in traditional (commercial airliners) and low-altitude (drones and eVTOL aircraft) airspace, an autonomous air traffic control system is needed to accommodate high density air traffic and ensure safe separation between aircraft. We propose a deep multi-agent reinforcement learning framewor…
Autonomous vehicles (AVs) are on the road. To safely and efficiently interact with other road participants, AVs have to accurately predict the behavior of surrounding vehicles and plan accordingly. Such prediction should be probabilistic, to address the uncertainties in human behavior. Such prediction should also be interactive, since the distribution over all possible trajectories of the predicted vehicle depends not only on historical information, but also on future plans of other vehicles that interact with it. To achieve such interaction-aware predictions, we propose a probabilistic predic…
Researches have been devoted to making connected and automated vehicles (CAVs) faster in different traffic scenarios. By using C-V2X or DSRC communication protocol, CAVs can work more effectively. In this paper, we compare these two communication protocols on CAVs in three different traffic scenarios including ramp merging, intersection, and platoon brake. It shows there is a trade-off between communication range and interval when leveraging C-V2X or DSRC for CAVs. The result can help support further application designs for CAV autonomously choosing communication protocols in different traffic…
