arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2104.10455cs.LGcs.AIcs.SYeess.SY
收藏

交通信号控制的强化学习:与商业系统的比较

Reinforcement Learning for Traffic Signal Control: Comparison with Commercial Systems

The Alan Turing Institute(艾伦·图灵研究所);University of Warwick(华威大学);Ecole Normale Superieure de Paris-Saclay(巴黎-萨克雷高等师范学校);University of Manchester(曼彻斯特大学)

Alvaro Cabrejas-Egea, Raymond Zhang, Neil Walton

摘要:

最近,智能交通系统正在利用增加的传感覆盖范围和计算能力来提供数据密集型解决方案,从而实现比传统系统更高的性能水平。在交通信号控制(TSC)中,这使得基于机器学习(ML)的系统得以出现。在这一组中,强化学习(RL)方法表现得特别好。由于缺乏用于TSC的ML行业标准,探索RL的文献通常缺乏与商业可用系统的比较以及代理如何操作的直接公式。在这里,我们试图弥合这一差距。我们提出了三种不同的架构TSC RL代理和比较它们对目前使用的商业系统MOVA,SurTrac和循环控制器,并提供伪代码。代理使用Deep Q-Learning和Actor Critic的变体,使用基于队列长度的状态和奖励。他们的表现进行了比较,在不同的地图场景与可变的需求,评估他们的全球延迟和平均队列长度。我们发现,与现有的商业系统相比,基于RL的系统可以显着和一致地实现较低的延迟。

英文摘要:

Recently, Intelligent Transportation Systems are leveraging the power of increased sensory coverage and computing power to deliver data-intensive solutions achieving higher levels of performance than traditional systems. Within Traffic Signal Control (TSC), this has allowed the emergence of Machine Learning (ML) based systems. Among this group, Reinforcement Learning (RL) approaches have performed particularly well. Given the lack of industry standards in ML for TSC, literature exploring RL often lacks comparison against commercially available systems and straightforward formulations of how the agents operate. Here we attempt to bridge that gap. We propose three different architectures for TSC RL agents and compare them against the currently used commercial systems MOVA, SurTrac and Cyclic controllers and provide pseudo-code for them. The agents use variations of Deep Q-Learning and Actor Critic, using states and rewards based on queue lengths. Their performance is compared in across different map scenarios with variable demand, assessing them in terms of the global delay and average queue length. We find that the RL-based systems can significantly and consistently achieve lower delays when compared with existing commercial systems.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑