arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2310.12359cs.MAcs.LG
收藏

MARVEL:面向大规模可变限速的多智能体强化学习

MARVEL: Multi-Agent Reinforcement-Learning for Large-Scale Variable Speed Limits

Institute for Software Integrated Systems, Vanderbilt University(范德堡大学软件集成系统研究所)

Yuhang Zhang, Marcos Quinones-Grueiro, Zhiyao Zhang, Yanbing Wang, William Barbour, Gautam Biswas, Daniel Work

AI总结:

提出MARVEL框架,利用多智能体强化学习实现大规模可变限速控制,在真实部署设置下通过参数共享和适应性奖励提升交通安全63.4%和机动性58.6%,并验证了可解释性与部署能力。

摘要:

可变限速(VSL)控制作为一种有前景的高速公路交通管理策略,已在全球范围内部署,能够根据实时交通状况动态调整限速,从而提升交通安全。迄今为止,大多数已部署的VSL控制算法基于规则,在多变且复杂的交通场景下缺乏泛化能力。在本工作中,我们提出了MARVEL(面向大规模可变限速的多智能体强化学习),一种适用于真实世界部署设置的高速公路走廊大规模VSL控制的新型框架。MARVEL仅利用现实世界中可观测的传感信息作为状态输入,并通过包含对交通状况适应性、安全性和机动性的奖励结构进行学习,从而实现多智能体协调。通过在所有VSL智能体之间共享参数,所提出的框架能够扩展以覆盖包含许多智能体的走廊。策略在微观交通仿真环境中训练,聚焦于一段包含8个VSL智能体、全长7英里的短高速公路路段。在测试阶段,这些策略被应用于美国田纳西州纳什维尔附近I-24公路上覆盖17英里、包含34个VSL智能体的更广泛网络。与无控制场景相比,基于MARVEL的方法将交通安全提升了63.4%,与已在I-24部署的现行最优算法相比,交通机动性提升了58.6%。此外,我们进行了可解释性分析,以检查智能体的决策过程,并探索在不同交通状况下学到的策略。最后,我们使用从I-24收集的真实世界数据测试了从基于仿真的实验中学习到的策略的响应,并展示了其部署能力。

英文摘要:

Variable Speed Limit (VSL) control acts as a promising highway traffic management strategy with worldwide deployment, which can enhance traffic safety by dynamically adjusting speed limits according to real-time traffic conditions. Most of the deployed VSL control algorithms so far are rule-based, lacking generalizability under varying and complex traffic scenarios. In this work, we propose MARVEL (Multi-Agent Reinforcement-learning for large-scale Variable spEed Limits), a novel framework for large-scale VSL control on highway corridors with real-world deployment settings. MARVEL utilizes only sensing information observable in the real world as state input and learns through a reward structure that incorporates adaptability to traffic conditions, safety, and mobility, thereby enabling multi-agent coordination. With parameter sharing among all VSL agents, the proposed framework scales to cover corridors with many agents. The policies are trained in a microscopic traffic simulation environment, focusing on a short freeway stretch with 8 VSL agents spanning 7 miles. For testing, these policies are applied to a more extensive network with 34 VSL agents spanning 17 miles of I-24 near Nashville, TN, USA. MARVEL-based method improves traffic safety by 63.4% compared to the no control scenario and enhances traffic mobility by 58.6% compared to a state-of-the-practice algorithm that has been deployed on I-24. Besides, we conduct an explainability analysis to examine the decision-making process of the agents and explore the learned policy under different traffic conditions. Finally, we test the response of the policy learned from the simulation-based experiments with real-world data collected from I-24 and illustrate its deployment capability.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑