arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2404.15696cs.RO
收藏

基于模型稳定性增强的延迟感知多智能体强化学习用于协同自适应巡航控制

Delay-Aware Multi-Agent Reinforcement Learning for Cooperative Adaptive Cruise Control with Model-based Stability Enhancement

Tongji University(同济大学);Purdue University(普渡大学)

Jiaqi Liu, Ziran Wang, Peng Hang, Jian Sun

AI总结:

针对CACC中的通信延迟问题,提出DAMARL框架,通过MADA-MDP、CTDE多智能体学习、注意力策略网络及速度优化动作过滤,提升车辆编队安全与稳定性。

摘要:

协同自适应巡航控制(CACC)是在网联自动驾驶车辆(CAV)系统中组织车辆编队行驶的典型控制策略,可显著提升交通效率并降低能耗。近年来,强化学习(RL)等数据驱动方法因其在效率和灵活性方面的显著优势,已被用于解决该任务。然而,真实CACC系统中经常出现的延迟问题,却很少被当前基于RL的方法考虑。为解决这一问题,我们提出了一种延迟感知多智能体强化学习(DAMARL)框架,旨在实现安全、稳定的CACC控制。我们使用多智能体延迟感知马尔可夫决策过程(MADA-MDP)对整个决策过程建模,并开发了一个集中式训练、分布式执行(CTDE)的MARL框架,用于CACC编队的分布式控制。为增强CAV通信与决策性能,引入了集成注意力机制的策略网络。此外,还加入了基于速度优化模型的动作过滤器,以进一步确保编队稳定性。在多种延迟条件和编队规模下的实验结果表明,我们的方法在编队安全性、稳定性和整体性能方面始终优于基线方法。

英文摘要:

Cooperative Adaptive Cruise Control (CACC) represents a quintessential control strategy for orchestrating vehicular platoon movement within Connected and Automated Vehicle (CAV) systems, significantly enhancing traffic efficiency and reducing energy consumption. In recent years, the data-driven methods, such as reinforcement learning (RL), have been employed to address this task due to their significant advantages in terms of efficiency and flexibility. However, the delay issue, which often arises in real-world CACC systems, is rarely taken into account by current RL-based approaches. To tackle this problem, we propose a Delay-Aware Multi-Agent Reinforcement Learning (DAMARL) framework aimed at achieving safe and stable control for CACC. We model the entire decision-making process using a Multi-Agent Delay-Aware Markov Decision Process (MADA-MDP) and develop a centralized training with decentralized execution (CTDE) MARL framework for distributed control of CACC platoons. An attention mechanism-integrated policy network is introduced to enhance the performance of CAV communication and decision-making. Additionally, a velocity optimization model-based action filter is incorporated to further ensure the stability of the platoon. Experimental results across various delay conditions and platoon sizes demonstrate that our approach consistently outperforms baseline methods in terms of platoon safety, stability and overall performance.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑