基于多Agent深度强化学习的网约车服务在线匹配优化
Optimizing Online Matching for Ride-Sourcing Services with Multi-Agent Deep Reinforcement Learning
摘要:
通过移动互联网有效地连接司机和乘客,乘车外包服务正在重塑人们的出行方式。空闲司机和等待乘客之间的在线匹配是乘车系统中最关键的组成部分之一。平均上车距离或时间是衡量系统效率的重要指标,因为它影响乘客的等待时间和司机的利用率。如果平台在匹配池中积累了更多的空闲司机和等待乘客,自然可以预期实现更有效的二分匹配(平均接车时间更短)。特定乘客请求也可以受益于延迟匹配,因为他/她可以在等待几秒钟之后与更近的空闲驾驶员匹配。受延迟匹配的潜在好处的启发,本文建立了一个两阶段的框架,该框架结合了组合优化和多智能体深度强化学习方法。采用多智能体强化学习方法动态确定每个乘客请求的延迟时间(或每个请求进入匹配池的时间),而组合优化则在匹配池中对空闲司机和等待乘客进行最优二分匹配。提出了时空多智能体深度Q学习(ST-M-DQN)和时空多智能体行动者-评论者(ST-M-A2 C)两种强化学习方法。通过大量的经验性实验与精心设计的模拟器,我们表明,所提出的框架是能够显着提高系统的性能。
英文摘要:
Ride-sourcing services are now reshaping the way people travel by effectively connecting drivers and passengers through mobile internets. Online matching between idle drivers and waiting passengers is one of the most key components in a ride-sourcing system. The average pickup distance or time is an important measurement of system efficiency since it affects both passengers' waiting time and drivers' utilization rate. It is naturally expected that a more effective bipartite matching (with smaller average pickup time) can be implemented if the platform accumulates more idle drivers and waiting passengers in the matching pool. A specific passenger request can also benefit from a delayed matching since he/she may be matched with closer idle drivers after waiting for a few seconds. Motivated by the potential benefits of delayed matching, this paper establishes a two-stage framework which incorporates a combinatorial optimization and multi-agent deep reinforcement learning methods. The multi-agent reinforcement learning methods are used to dynamically determine the delayed time for each passenger request (or the time at which each request enters the matching pool), while the combinatorial optimization conducts an optimal bipartite matching between idle drivers and waiting passengers in the matching pool. Two reinforcement learning methods, spatio-temporal multi-agent deep Q learning (ST-M-DQN) and spatio-temporal multi-agent actor-critic (ST-M-A2C) are developed. Through extensive empirical experiments with a well-designed simulator, we show that the proposed framework is able to remarkably improve system performances.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。