arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2103.16223cs.LGcs.SYeess.SY
收藏

面向交通信号控制强化学习的实际部署

Towards Real-World Deployment of Reinforcement Learning for Traffic Signal Control

Fraunhofer IOSB-INA(弗劳恩霍夫IOSB-INA);Stührenberg GmbH(施蒂伦贝格有限公司);OWL University of Applied Sciences and Arts(OWL应用科学与艺术大学);University of Groningen(格罗宁根大学)

Arthur Müller, Vishal Rangras, Georg Schnittker, Michael Waldmann, Maxim Friesen, Tobias Ferfers, Lukas Schreckenberg, Florian Hufen, Jürgen Jasperneite, Marco Wiering

AI总结:

针对交通信号控制强化学习研究多依赖简化仿真、难落地的问题,提出适配德国莱姆戈真实场景、符合安全规范且兼容OpenAI gym接口的基准工具LemgoRL,通过分布式CPU训练验证了其有效性,推动RL算法实际部署。

摘要:

交叉口交通信号控制器(TSC)的次优控制策略会加剧交通拥堵,并对人类健康和环境造成负面影响。用于交通信号控制的强化学习(RL)是设计更优控制策略的一种极具前景的方法,近年来已吸引了大量研究兴趣。然而,该领域的大多数研究都采用简化的交通场景仿真环境来训练基于RL的TSC。要将RL部署到实际交通系统中,必须弥合简化仿真环境与实际应用之间的差距。为此,我们提出了LemgoRL——一种在德国中等城市莱姆戈(Lemgo)的高保真仿真环境中训练作为TSC的RL智能体的基准工具。除了高保真仿真模型外,LemgoRL还包含一个交通信号逻辑单元,可确保符合所有监管和安全要求。LemgoRL提供与知名工具包OpenAI gym相同的接口,便于在现有研究工作中部署。为验证LemgoRL的功能与适用性,我们利用分布式并行RL框架在CPU集群上训练了一种最先进的深度RL算法,并将其性能与其他方法进行了对比。我们的基准工具将推动RL算法向实际应用方向发展。

英文摘要:

Sub-optimal control policies in intersection traffic signal controllers (TSC) contribute to congestion and lead to negative effects on human health and the environment. Reinforcement learning (RL) for traffic signal control is a promising approach to design better control policies and has attracted considerable research interest in recent years. However, most work done in this area used simplified simulation environments of traffic scenarios to train RL-based TSC. To deploy RL in real-world traffic systems, the gap between simplified simulation environments and real-world applications has to be closed. Therefore, we propose LemgoRL, a benchmark tool to train RL agents as TSC in a realistic simulation environment of Lemgo, a medium-sized town in Germany. In addition to the realistic simulation model, LemgoRL encompasses a traffic signal logic unit that ensures compliance with all regulatory and safety requirements. LemgoRL offers the same interface as the wellknown OpenAI gym toolkit to enable easy deployment in existing research work. To demonstrate the functionality and applicability of LemgoRL, we train a state-of-the-art Deep RL algorithm on a CPU cluster utilizing a framework for distributed and parallel RL and compare its performance with other methods. Our benchmark tool drives the development of RL algorithms towards real-world applications.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑