arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2407.14775eess.SYcs.SY
收藏

强化学习交通信号控制中的相位再服务

Phase Re-service in Reinforcement Learning Traffic Signal Control

Zhiyao Zhang, George Gunter, Marcos Quinones-Grueiro, Yuhang Zhang, William Barbour, Gautam Biswas, Daniel Work

AI总结:

本文提出在强化学习交通信号控制中引入基于冲击波理论的相位再服务机制,将问题建模为SMDP并用PPO求解,实验表明该方法显著降低车辆延误和停车次数。

摘要:

本文提出了一种将相位再服务与强化学习(RL)相结合的交通信号控制新方法。强化学习智能体直接确定预定义序列中下一相位的持续时间。在强化学习智能体的决策被执行之前,我们使用冲击波理论来估计被允许再服务的指定流向处的队列扩展,并判断是否有必要进行相位再服务。如有必要,则在下一个常规相位之前插入一个临时相位再服务。我们将强化学习问题形式化为半马尔可夫决策过程(SMDP),并使用近端策略优化(PPO)进行求解。我们进行了一系列实验,结果表明引入相位再服务带来了显著改进。车辆延误的平均值最多降低29.95%,标准差最多降低59.21%。停车次数平均减少26.05%,标准差降低45.77%。

英文摘要:

This article proposes a novel approach to traffic signal control that combines phase re-service with reinforcement learning (RL). The RL agent directly determines the duration of the next phase in a pre-defined sequence. Before the RL agent's decision is executed, we use the shock wave theory to estimate queue expansion at the designated movement allowed for re-service and decide if phase re-service is necessary. If necessary, a temporary phase re-service is inserted before the next regular phase. We formulate the RL problem as a semi-Markov decision process (SMDP) and solve it with proximal policy optimization (PPO). We conducted a series of experiments that showed significant improvements thanks to the introduction of phase re-service. Vehicle delays are reduced by up to 29.95% of the average and up to 59.21% of the standard deviation. The number of stops is reduced by 26.05% on average with 45.77% less standard deviation.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑