In this paper, we evaluate the use of Reinforcement Learning (RL) to solve a classic combinatorial optimization problem: the Capacitated Vehicle Routing Problem (CVRP). We formalize this problem in the RL framework and compare two of the most promising RL approaches with traditional solving techniques on a set of benchmark instances. We measure the different approaches with the quality of the solution returned and the time required to return it. We found that despite not returning the best solution, the RL approach has many advantages over traditional solvers. First, the versatility of the framework allows the resolution of more complex combinatorial problems. Moreover, instead of trying to solve a specific instance of the problem, the RL algorithm learns the skills required to solve the problem. The trained policy can then quasi instantly provide a solution to an unseen problem without having to solve it from scratch. Finally, the use of trained models makes the RL solver by far the fastest, and therefore make this approach more suited for commercial use where the user experience is paramount. Techniques like Knowledge Transfer can also be used to improve the training efficiency of the algorithm and help solve bigger and more complex problems.
Refining Hybrid Genetic Search for CVRP via Reinforcement Learning-Finetuned LLM
School of Teacher Education, Nanjing University of Information Science and Technology, China（南京信息工程大学教师教育学院）；Nanyang Technological University, Singapore（新加坡南洋理工大学）；Singapore Management University, Singapore（新加坡管理大学）
Deep Reinforcement Learning for Solving the Heterogeneous Capacitated Vehicle Routing Problem
National University of Singapore（新加坡国立大学）；Singapore Institute of Manufacturing Technology（新加坡制造技术研究院）；Southwest Jiaotong University（西南交通大学）；Shandong University（山东大学）；Nanyang Technological University（南洋理工大学）
Solve routing problems with a residual edge-graph attention neural network
Southwest Jiaotong University（西南交通大学）；Technology and Equipment of Rail Transit Operation and Maintenance Key Laboratory of Sichuan Province（四川省轨道交通运维技术与装备重点实验室）；Auburn University at Montgomery（奥本大学蒙哥马利分校）
Diversity Optimization for Travelling Salesman Problem via Deep Reinforcement Learning
South China University of Technology（华南理工大学）；Singapore Management University（新加坡管理大学）；Massachusetts Institute of Technology（麻省理工学院）；Eindhoven University of Technology（埃因霍温理工大学）
Routing algorithms as tools for integrating social distancing with emergency evacuation
Stanford University（斯坦福大学）；Woods Institute for the Environment, Stanford University（斯坦福大学伍兹环境研究所）；Institute for Computational and Mathematical Engineering, Stanford University（斯坦福大学计算与数学工程研究所）；Department of Biology, Stanford University（斯坦福大学生物系）；Department of Earth System Science, Stanford University（斯坦福大学地球系统科学系）
Hierarchical Learning-based Graph Partition for Large-scale Vehicle Routing Problems
The Hong Kong University of Science and Technology（香港科技大学）；University of Oxford（牛津大学）；University of Alberta（阿尔伯塔大学）；Singapore Management University（新加坡管理大学）
Bad-Policy Density: A Measure of Reinforcement Learning Hardness
DeepMind；Department of Computer Science, Brown University（布朗大学计算机科学系）；Department of Computer Science, Stanford University（斯坦福大学计算机科学系）；Department of Computer Science, Carnegie Mellon University（卡内基梅隆大学计算机科学系）；Khoury College of Computer Sciences, Northeastern University（东北大学Khoury计算机科学学院）
Baseline-Free Policy Optimization for Neural Combinatorial Optimization
Facultad de Ingeniería y Ciencias, Universidad Adolfo Ibáñez（阿道夫·伊瓦涅斯大学工程与科学学院）；Dirección de Programas, Investigación y Desarrollo, Armada de Chile（智利海军计划、研究与发展局）；Millennium Nucleus for Social Data Science (SODAS)（千禧年社会数据科学核心（SODAS））；Millennium Nucleus in Data Science for Plant Resilience (PhytoLearning)（千禧年植物韧性数据科学核心（PhytoLearning））
Adversarial Generative Flow Network for Solving Vehicle Routing Problems
Singapore Management University（新加坡管理大学）；Singapore Institute of Manufacturing Technology (SIMTech), Agency of Science, Technology and Research (A*STAR)（新加坡制造技术研究院（新加坡科技研究局下属机构））
Synergizing Reinforcement Learning and Genetic Algorithms for Neural Combinatorial Optimization
C 2 DL, Institute of Automation, Chinese Academy of Sciences（中国科学院自动化研究所）；School of Artificial Intelligence, University of Chinese Academy of Sciences（中国科学院大学人工智能学院）；School of Advanced Interdisciplinary Sciences, University of Chinese Academy of Sciences（中国科学院大学先进交叉学科学院）；Department of Computer Science and Technology, Tsinghua University（清华大学计算机科学与技术系）
Reinforcement Learning for Graph Generation under a Hard Assortativity Constraint
School of Computational Sciences, Korea Institute for Advanced Study（韩国高等科学研究院计算科学系）；Department of Physics Education, Seoul National University（首尔国立大学物理教育系）；Center for Theoretical Physics and Artificial Intelligence Institute, Seoul National University（首尔国立大学理论物理与人工智能研究所）；Center for AI and Natural Sciences, Korea Institute for Advanced Study（韩国高等科学研究院人工智能与自然科学中心）
Blue River Controls: A toolkit for Reinforcement Learning Control Systems on Hardware
University Of Calgary（卡尔加里大学）；Blue River Technology（蓝河科技）
我们为Quanser的硬件在环软件开发工具包（HIL SDK）提供了一个简单的硬件包装器，以便轻松开发新的Quanser硬件。要连接到硬件，我们使用Cython编写的模块。内部QuanserWrapper类处理与硬件交互的大多数困难方面，包括定时（使用硬件定时器），并确保发送到硬件的数据是安全和正确的，其中安全对应于指定硬件的安全工作电压和电流。最近强化学习（RL）的大部分成功都是通过OpenAI Gym和Deepmind Control Suite等训练和测试工具实现的。不幸的是，用于快速测试和将高频RL算法从仿真转移到真实硬件环境的工具仍然很少。我们介绍了Blue River Controls，这是一个允许在真实硬件上训练和测试强化学习算法的工具。它具有基于OpenAI Gym的简单界面，可直接在仿真和硬件上工作。我们使用Quanser的Qube Servo 2-USB平台，一种欠驱动旋转摆作为初始测试设备。我们还提供工具来简化其他硬件上的RL算法训练。包括来自经典控制器和预先训练的RL代理的几个基线，以比较任务之间的性能。Blue River Controls可在此https URL获得：https://github.com/BlueRiverTech/quanser-openai-driver
The unknotting number, hard unknot diagrams, and reinforcement learning
Google DeepMind, London, UK（谷歌深思（DeepMind），伦敦，英国）；Mathematical Institute, University of Oxford, Andrew Wiles Building, Radcliffe Observatory Quarter, Woodstock Road, Oxford, OX2 6GG, UK（牛津大学数学研究所）
