arXiv 2201.006952022-01cs.IRcs.AIcs.LG

基于学习排序精馏的高效组合优化模型

An Efficient Combinatorial Optimization Model Using Learning-to-Rank Distillation

Sungkyunkwan University（成均馆大学）

Honguk Woo, Hyunsung Lee, Sangwoo Cho

备注：: Accepted at AAAI 22, 13 pages

摘要：

最近，深度强化学习（RL）已经证明了其在解决组合优化问题（COP）中的可行性。学习排序技术是信息检索领域的研究热点。虽然几个COP可以被公式化为输入项的优先级排序，这在信息检索中是常见的，但还没有充分探索如何将学习排序技术结合到COP的深度RL中。在本文中，我们提出了基于学习排序蒸馏的COP框架，其中通过RL获得的COP的高性能排名策略可以被蒸馏成非迭代的简单模型，从而实现低延迟的COP求解器。具体来说，我们采用近似的排名蒸馏，使基于分数的排名模型通过梯度下降学习。此外，我们使用有效的序列采样，以提高推理性能与有限的延迟。通过该框架，我们证明了一个蒸馏模型不仅可以实现与其各自的高性能RL相当的性能，而且还可以提供几倍的推理速度。我们评估的框架与几个COP，如基于优先级的任务调度和多维背包，展示了推理延迟和性能方面的框架的好处。

英文摘要：

Recently, deep reinforcement learning (RL) has proven its feasibility in solving combinatorial optimization problems (COPs). The learning-to-rank techniques have been studied in the field of information retrieval. While several COPs can be formulated as the prioritization of input items, as is common in the information retrieval, it has not been fully explored how the learning-to-rank techniques can be incorporated into deep RL for COPs. In this paper, we present the learning-to-rank distillation-based COP framework, where a high-performance ranking policy obtained by RL for a COP can be distilled into a non-iterative, simple model, thereby achieving a low-latency COP solver. Specifically, we employ the approximated ranking distillation to render a score-based ranking model learnable via gradient descent. Furthermore, we use the efficient sequence sampling to improve the inference performance with a limited delay. With the framework, we demonstrate that a distilled model not only achieves comparable performance to its respective, high-performance RL, but also provides several times faster inferences. We evaluate the framework with several COPs such as priority-based task scheduling and multidimensional knapsack, demonstrating the benefits of the framework in terms of inference latency and performance.

机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。

摘要：

英文摘要：

相关论文