基于时空超图的多智能体强化学习交通信号控制
Towards Multi-agent Reinforcement Learning based Traffic Signal Control through Spatio-temporal Hypergraphs
AI总结:
针对传统交通信号控制无法捕捉路口高阶动态时空关联的问题,提出融合超图与时空图学习的MA-SAC多智能体强化学习框架,有效降低了车辆平均出行时间并维持了高吞吐量。
摘要:
交通信号控制系统(TSCSs)是智能交通管理的重要组成部分,有助于促进车辆高效通行。传统方法通常将道路网络简化为标准图,这导致无法考虑相邻交叉路口交通数据的动态特性,从而忽略了实时控制所需的高阶互连。为了解决这一问题,我们提出了一种新颖的TSCS框架以实现智能交通控制。该框架与多个相邻的边缘计算服务器协同工作,以收集整个道路网络的交通信息。为了提升交通信号控制的效率,我们设计了一种多智能体软执行者-评论者(MA-SAC)强化学习算法。在该算法中,每个交叉路口部署了独立的智能体,其任务是共同优化整个道路网络的交通流。此外,我们将超图学习引入MA-SAC的critic网络中,以实现道路网络中多个交叉路口的时空交互。该方法融合超图和时空图结构来编码交通数据,并捕捉多个交叉路口之间复杂的时空相关性。我们在多种数据集上进行的实证评估表明,该框架在最小化平均车辆出行时间和维持高吞吐量性能方面具有优越性。这项工作促进了更智能的城市交通管理解决方案的发展。我们在 https://github.com/Edun-Eyes/TSC 发布了代码以支持本研究的可复现性。
英文摘要:
Traffic signal control systems (TSCSs) are integral to intelligent traffic management, fostering efficient vehicle flow. Traditional approaches often simplify road networks into standard graphs, which results in a failure to consider the dynamic nature of traffic data at neighboring intersections, thereby neglecting higher-order interconnections necessary for real-time control. To address this, we propose a novel TSCS framework to realize intelligent traffic control. This framework collaborates with multiple neighboring edge computing servers to collect traffic information across the road network. To elevate the efficiency of traffic signal control, we have crafted a multi-agent soft actor-critic (MA-SAC) reinforcement learning algorithm. Within this algorithm, individual agents are deployed at each intersection with a mandate to optimize traffic flow across the road network collectively. Furthermore, we introduce hypergraph learning into the critic network of MA-SAC to enable the spatio-temporal interactions from multiple intersections in the road network. This method fuses hypergraph and spatio-temporal graph structures to encode traffic data and capture the complex spatio-temporal correlations between multiple intersections. Our empirical evaluation, tested on varied datasets, demonstrates the superiority of our framework in minimizing average vehicle travel times and sustaining high-throughput performance. This work facilitates the development of more intelligent urban traffic management solutions. We release the code to support the reproducibility of this work at https://github.com/Edun-Eyes/TSC
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。