面向可变需求自适应的自主路由与取件问题的多智能体强化学习
Multiagent Reinforcement Learning for Autonomous Routing and Pickup Problem with Adaptation to Variable Demand
AI总结：
本文提出一种多智能体强化学习框架，结合在线博弈与基于Wasserstein模糊集有效性区域的离线近似切换机制，使自主车辆路由取件策略能适应城市高峰/非高峰需求波动，并在旧金山真实出租车数据上验证其优于现有强化学习和运筹学方法。
摘要：
我们推导出一个学习框架，用于为一组自主车辆生成路由/取件策略，这些车辆的任务是在城市地图上服务随机出现的请求。我们关注的策略具有以下特点：1）能够在车辆之间产生协调，从而减少服务请求的等待时间；2）是非短视的，并考虑先验的潜在未来请求；3）能够适应底层需求分布的变化。具体而言，我们感兴趣的是能够适应城市环境中实际需求条件波动的策略，例如高峰时段与非高峰时段。我们通过以下组合来实现这一目标：（i）一种在线博弈算法，用于提高离线训练策略的性能，以及（ii）一种离线近似方案，允许适应底层需求模型的变化。特别是，我们通过使用Wasserstein模糊集的q-valid半径来量化有效性区域，从而使学习到的策略能够适应不同的需求分布。我们提出了一种机制，当当前需求超出原始有效性区域时，切换最初训练的离线近似。在这种情况下，我们建议使用一种离线架构，该架构基于在Wasserstein距离上更接近当前需求的历史需求模型进行训练。我们在旧金山的真实出租车请求上学习路由和取件策略，这些请求在高峰时段和非高峰时段之间具有高度可变性，证明了我们的方法能够适应需求分布的真实波动。我们的数值结果表明，我们的方法优于基于rollout的替代强化学习方案，以及运筹学中的其他经典方法。
英文摘要：
We derive a learning framework to generate routing/pickup policies for a fleet of autonomous vehicles tasked with servicing stochastically appearing requests on a city map. We focus on policies that 1) give rise to coordination amongst the vehicles, thereby reducing wait times for servicing requests, 2) are non-myopic, and consider a-priori potential future requests, 3) can adapt to changes in the underlying demand distribution. Specifically, we are interested in policies that are adaptive to fluctuations of actual demand conditions in urban environments, such as on-peak vs. off-peak hours. We achieve this through a combination of (i) an online play algorithm that improves the performance of an offline-trained policy, and (ii) an offline approximation scheme that allows for adapting to changes in the underlying demand model. In particular, we achieve adaptivity of our learned policy to different demand distributions by quantifying a region of validity using the q-valid radius of a Wasserstein Ambiguity Set. We propose a mechanism for switching the originally trained offline approximation when the current demand is outside the original validity region. In this case, we propose to use an offline architecture, trained on a historical demand model that is closer to the current demand in terms of Wasserstein distance. We learn routing and pickup policies over real taxicab requests in San Francisco with high variability between on-peak and off-peak hours, demonstrating the ability of our method to adapt to real fluctuation in demand distributions. Our numerical results demonstrate that our method outperforms alternative rollout-based reinforcement learning schemes, as well as other classical methods from operations research.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
