异构排队系统中作业路由的高效强化学习
Efficient Reinforcement Learning for Routing Jobs in Heterogeneous Queueing Systems
AI总结：
针对异构多服务器排队系统作业路由问题，提出基于策略梯度的ACHQ算法，采用低维软阈值参数化利用排队结构，在双服务器特例下收敛至近似全局最优，期望响应时间较贪婪策略改善约30%。
摘要：
我们考虑将到达中央队列的作业高效路由到异构服务器系统的问题。与同构系统不同，已知阈值策略（当队列长度超过特定阈值时将作业路由到慢速服务器）对于一快一慢的双服务器系统是最优的。但多服务器系统的最优策略未知且难以寻找。虽然强化学习（RL）在此类情况下学习策略具有巨大潜力，但我们的问题具有指数级大的状态空间，导致标准RL效率低下。在本工作中，我们提出ACHQ，一种基于策略梯度的高效算法，采用低维软阈值策略参数化，利用底层排队结构。我们为一般情况提供驻点收敛保证，并证明尽管采用低维参数化，ACHQ在双服务器特例下收敛至近似全局最优。仿真表明，相比路由至最快可用服务器的贪婪策略，期望响应时间改善达约30%。
英文摘要：
We consider the problem of efficiently routing jobs that arrive into a central queue to a system of heterogeneous servers. Unlike homogeneous systems, a threshold policy, that routes jobs to the slow server(s) when the queue length exceeds a certain threshold, is known to be optimal for the one-fast-one-slow two-server system. But an optimal policy for the multi-server system is unknown and non-trivial to find. While Reinforcement Learning (RL) has been recognized to have great potential for learning policies in such cases, our problem has an exponentially large state space size, rendering standard RL inefficient. In this work, we propose ACHQ, an efficient policy gradient based algorithm with a low dimensional soft threshold policy parameterization that leverages the underlying queueing structure. We provide stationary-point convergence guarantees for the general case and despite the low-dimensional parameterization prove that ACHQ converges to an approximate global optimum for the special case of two servers. Simulations demonstrate an improvement in expected response time of up to ~30% over the greedy policy that routes to the fastest available server.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
