arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2503.02189cs.MA
收藏

基于多智能体强化学习的自适应交通信号控制:模拟真实世界走廊的案例研究

Adaptive Traffic Signal Control based on Multi-Agent Reinforcement Learning. Case Study on a simulated real-world corridor

Dickness Kakitahi Kwesiga, Angshuman Guin, Michael Hunter

AI总结:

本文提出一种集中训练分散执行的多智能体近端策略优化(MA-PPO)算法,用于真实世界干线走廊的自适应协调信号控制,并在七交叉口仿真中相较感应协调控制显著降低行程时间,展现出良好的稳定性与适应性。

摘要:

以往为自适应交通信号控制构建多智能体强化学习(RL)算法的研究主要使用基于值函数的RL方法。然而,近期文献表明,基于策略的方法在部分可观测环境中可能表现更好。此外,由于文献中常见的简化假设,RL方法在很大程度上仍未针对真实世界的常规信号配时方案进行测试。本研究试图弥补这些不足,并构建了一种多智能体近端策略优化(MA-PPO)算法,以在干线走廊上实现自适应和协调的交通控制。所构建的MA-PPO采用集中训练与分散执行框架下的集中式评论家架构。智能体被设计为允许选择并实施最多八个信号相位,这与现场控制器中常见的实现方式一致。所构建的算法在一个模拟的真实世界七交叉口走廊上进行了测试。研究发现,每个智能体的收敛速度取决于动作空间的大小,而动作空间的大小又取决于信号相位的数量和顺序。将所构建的MA-PPO自适应控制算法的性能与现场实施的感应协调信号控制(ASC)进行了比较,后者使用PTV-Vissim-MaxTime软件在环仿真(SILs)进行建模。训练后的MA-PPO在所有流向上的表现均显著优于ASC。与ASC相比,MA-PPO在主协调方向和次协调方向上的行程时间分别改善了2%和24%。对于横向街道流向,MA-PPO也表现出显著的过街时间缩短。流量敏感性实验表明,所构建的MA-PPO对交通需求变化表现出良好的稳定性、鲁棒性和适应性。

英文摘要:

Previous studies that have formulated multi-agent reinforcement learning (RL) algorithms for adaptive traffic signal control have primarily used value-based RL methods. However, recent literature has shown that policy-based methods may perform better in partially observable environments. Additionally, RL methods remain largely untested for real-world normally signal timing plans because of the simplifying assumptions common in the literature. The current study attempts to address these gaps and formulates a multi-agent proximal policy optimization (MA-PPO) algorithm to implement adaptive and coordinated traffic control along an arterial corridor. The formulated MA-PPO has a centralized-critic architecture under a centralized training and decentralized execution framework. Agents are designed to allow selection and implementation of up to eight signal phases, as commonly implemented in field controllers. The formulated algorithm is tested on a simulated real-world seven intersection corridor. The speed of convergence for each agent was found to depend on the size of the action space, which depends on the number and sequence of signal phases. The performance of the formulated MA-PPO adaptive control algorithm is compared with the field implemented actuated-coordinated signal control (ASC), modeled using PTV-Vissim-MaxTime software in the loop simulation (SILs). The trained MA-PPO performed significantly better than the ASC for all movements. Compared to ASC the MA-PPO showed 2% and 24% improvements in travel time in the primary and secondary coordination directions, respectively. For cross streets movements MA-PPO also showed significant crossing time reductions. Volume sensitivity experiments revealed that the formulated MA-PPO demonstrated good stability, robustness, and adaptability to changes in traffic demand.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑