arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2202.07147eess.SYcs.SY
收藏

基于图元强化学习的可迁移自主按需移动

Graph Meta-Reinforcement Learning for Transferable Autonomous Mobility-on-Demand

Daniele Gammelli, Kaidi Yang, James Harrison, Filipe Rodrigues, Francisco C. Pereira, Marco Pavone

摘要:

自主移动按需(AMoD)系统代表了现有交通模式的一种有吸引力的替代方案,目前受到城市化和日益增长的旅行需求的挑战。通过集中控制自动驾驶车队,这些系统为客户提供移动服务,目前已开始在全球多个城市部署。当前用于控制AMoD系统的基于学习的方法仅限于单个城市场景,由此允许服务运营商在同一交通系统内采取无限量的运营决策。然而,现实世界的系统运营商几乎无法负担为他们运营的每个城市完全重新培训AMoD控制器的费用,因为这可能导致培训期间大量的低质量决策,使单一城市策略成为一个潜在的不切实际的解决方案。为了解决这些限制,我们建议通过元强化学习(元RL)的镜头来形式化多城市AMoD问题,并设计一个基于递归图神经网络的演员-评论家算法。在我们的方法中,AMoD控制器被明确地训练,使得在新城市中的少量经验将产生良好的系统性能。从经验上讲,我们展示了通过元RL学习的控制策略如何能够通过学习快速适应的策略在看不见的城市中实现接近最优的性能,从而使它们不仅对新环境更具鲁棒性,而且对现实世界运营中常见的分布变化更具鲁棒性,例如特殊事件,意外拥堵和动态定价方案。

英文摘要:

Autonomous Mobility-on-Demand (AMoD) systems represent an attractive alternative to existing transportation paradigms, currently challenged by urbanization and increasing travel needs. By centrally controlling a fleet of self-driving vehicles, these systems provide mobility service to customers and are currently starting to be deployed in a number of cities around the world. Current learning-based approaches for controlling AMoD systems are limited to the single-city scenario, whereby the service operator is allowed to take an unlimited amount of operational decisions within the same transportation system. However, real-world system operators can hardly afford to fully re-train AMoD controllers for every city they operate in, as this could result in a high number of poor-quality decisions during training, making the single-city strategy a potentially impractical solution. To address these limitations, we propose to formalize the multi-city AMoD problem through the lens of meta-reinforcement learning (meta-RL) and devise an actor-critic algorithm based on recurrent graph neural networks. In our approach, AMoD controllers are explicitly trained such that a small amount of experience within a new city will produce good system performance. Empirically, we show how control policies learned through meta-RL are able to achieve near-optimal performance on unseen cities by learning rapidly adaptable policies, thus making them more robust not only to novel environments, but also to distribution shifts common in real-world operations, such as special events, unexpected congestion, and dynamic pricing schemes.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑