We present a novel deep reinforcement learning method to learn construction heuristics for vehicle routing problems. In specific, we propose a Multi-Decoder Attention Model (MDAM) to train multiple diverse policies, which effectively increases the chance of finding good solutions compared with existing methods that train only one policy. A customized beam search strategy is designed to fully exploit the diversity of MDAM. In addition, we propose an Embedding Glimpse layer in MDAM based on the recursive nature of construction, which can improve the quality of each policy by providing more informative embeddings. Extensive experiments on six different routing problems show that our method significantly outperforms the state-of-the-art deep learning based models.
Rethinking Light Decoder-based Solvers for Vehicle Routing Problems
School of Computing and Information Systems, Singapore Management University（新加坡管理大学计算与信息系统学院）；College of Computing and Data Science, Nanyang Technological University（南洋理工大学计算与数据科学学院）
Suleyman Demirel University（苏莱曼·德米雷尔大学）；UNIST（蔚山科学技术院）；Korea University（高丽大学）；Amazon（亚马逊）；University of South Florida（南佛罗里达大学）
Reinforcement learning has recently shown promise in learning quality solutions in many combinatorial optimization problems. In particular, the attention-based encoder-decoder models show high effectiveness on various routing problems, including the Traveling Salesman Problem (TSP). Unfortunately, they perform poorly for the TSP with Drone (TSP-D), requiring routing a heterogeneous fleet of vehicles in coordination -- a truck and a drone. In TSP-D, the two vehicles are moving in tandem and may need to wait at a node for the other vehicle to join. State-less attention-based decoder fails to mak…
Improving Generalization of Neural Vehicle Routing Problem Solvers Through the Lens of Model Architecture
Key Laboratory of Symbolic Computation and Knowledge Engineering of Ministry of Education, College of Computer Science and Technology, Jilin University（吉林大学计算机科学与技术学院、教育部符号计算与知识工程重点实验室）；Joint NTU-UBC Research Centre of Excellence in Active Living for the Elderly, Nanyang Technological University（南洋理工大学、新加坡国立大学-不列颠哥伦比亚大学老年人积极生活联合卓越研究中心）；WeBank-NTU Joint Research Institute on Fintech, Nanyang Technological University（南洋理工大学微众银行金融科技联合研究院）；College of Computing and Data Science, Nanyang Technological University（南洋理工大学计算与数据科学学院）
Rethinking Constraint Awareness for Efficient State Embedding of Neural Routing Solver
College of Computer Science and Software Engineering, Shenzhen University（深圳大学计算机科学与软件工程学院）；School of Automation and Intelligent Manufacturing, Southern University of Science and Technology（南方科技大学自动化与智能制造学院）；Guangdong Provincial Key Laboratory of Fully Actuated System Control Theory and Technology, Southern University of Science and Technology（广东省全驱动系统控制理论与技术重点实验室，南方科技大学）；Pengcheng Laboratory（鹏城实验室）
Department of Computer Science, Sun Yat-sen University（中山大学计算机科学系）；Sun Yat-sen University（中山大学）
Recent researches show that machine learning has the potential to learn better heuristics than the one designed by human for solving combinatorial optimization problems. The deep neural network is used to characterize the input instance for constructing a feasible solution incrementally. Recently, an attention model is proposed to solve routing problems. In this model, the state of an instance is represented by node features that are fixed over time. However, the fact is, the state of an instance is changed according to the decision that the model made at different construction steps, and the …
SNCF, the French public train company, is experimenting to develop new types of transportation services by tackling vehicle routing problems. While many deep learning models have been used to tackle efficiently vehicle routing problems, it is difficult to take into account time related constraints. In this paper, we solve the Capacitated Vehicle Routing Problem with Time Windows (CVRPTW) and the Capacitated Pickup and Delivery Problem with Time Windows (CPDPTW) with a constructive iterative Deep Learning algorithm. We use an Attention Encoder-Decoder structure and design a novel insertion heur…
Learning to Handle Complex Constraints for Vehicle Routing Problems
Nanyang Technological University（南洋理工大学）；Shandong University（山东大学）；Singapore Management University（新加坡管理大学）；Eindhoven University of Technology（埃因霍温理工大学）
Learning Graph-Indexed Trajectory Patterns for Stochastic On-Time Arrival Routing
Sichuan University-Pittsburgh Institute（四川大学匹兹堡研究院）；Sichuan University（四川大学）；College of Computer Science（计算机科学学院）；University College Dublin（都柏林大学）；School of Electrical and Electronic Engineering（电子与电气工程学院）；School of Electronics and Information Engineering（电子与信息工程学院）
GASE: Graph Attention Sampling with Edges Fusion for Solving Vehicle Routing Problems
School of Computer Science, University of Nottingham Ningbo China（宁波诺丁汉大学计算机科学学院）；School of Computer Science, University of Nottingham（诺丁汉大学计算机科学学院）；School of Electronics and Information Engineering, Tongji University（同济大学电子与信息工程学院）
National University of Singapore（新加坡国立大学）；Singapore Institute of Manufacturing Technology（新加坡制造技术研究院）；Southwest Jiaotong University（西南交通大学）；Shandong University（山东大学）；Nanyang Technological University（南洋理工大学）
Existing deep reinforcement learning (DRL) based methods for solving the capacitated vehicle routing problem (CVRP) intrinsically cope with homogeneous vehicle fleet, in which the fleet is assumed as repetitions of a single vehicle. Hence, their key to construct a solution solely lies in the selection of the next node (customer) to visit excluding the selection of vehicle. However, vehicles in real-world scenarios are likely to be heterogeneous with different characteristics that affect their capacity (or travel speed), rendering existing DRL methods less effective. In this paper, we tackle he…
Integrated Order Dispatching and Routing for Last-Mile Pickup via Deep Reinforcement Learning
College of Management and Economics, Tianjin University（天津大学管理与经济学部）；Laboratory of Computation and Analytics of Complex Management Systems (CACMS), Tianjin University（天津大学复杂管理系统计算与分析实验室）
This paper presents an approach to learn the local-search heuristics that iteratively improves the solution of Vehicle Routing Problem (VRP). A local-search heuristics is composed of a destroy operator that destructs a candidate solution, and a following repair operator that rebuilds the destructed one into a new one. The proposed neural network, as trained through actor-critic framework, consists of an encoder in form of a modified version of Graph Attention Network where node embeddings and edge embeddings are integrated, and a GRU-based decoder rendering a pair of destroy and repair operato…
Department of Psychology（心理学系）；Department of Computer Science（计算机科学系）；Stony Brook University（石溪大学）
The visual system processes a scene using a sequence of selective glimpses, each driven by spatial and object-based attention. These glimpses reflect what is relevant to the ongoing task and are selected through recurrent processing and recognition of the objects in the scene. In contrast, most models treat attention selection and recognition as separate stages in a feedforward process. Here we show that using capsule networks to create an object-centric hidden representation in an encoder-decoder model with iterative glimpse attention yields effective integration of attention and recognition.…
Efficient Decoder Scaling Strategy for Neural Routing Solvers
Guangdong Provincial Key Laboratory of Fully Actuated System Control Theory（广东省全驱动系统控制理论与技术重点实验室）；School of Automation（自动化学院）；Intelligent Manufacturing, Southern University of Science（智能制造，南方科技大学）；Department of Computer Science, City University of Hong Kong, Hong Kong SAR, China（计算机科学系，香港城市大学，香港特别行政区，中国）
Southwest Jiaotong University（西南交通大学）；Technology and Equipment of Rail Transit Operation and Maintenance Key Laboratory of Sichuan Province（四川省轨道交通运维技术与装备重点实验室）；Auburn University at Montgomery（奥本大学蒙哥马利分校）
For NP-hard combinatorial optimization problems, it is usually difficult to find high-quality solutions in polynomial time. The design of either an exact algorithm or an approximate algorithm for these problems often requires significantly specialized knowledge. Recently, deep learning methods provide new directions to solve such problems. In this paper, an end-to-end deep reinforcement learning framework is proposed to solve this type of combinatorial optimization problems. This framework can be applied to different problems with only slight changes of input (for example, for a traveling sale…
A Unified Model for Multi-Task Drone Routing in Post-Disaster Road Assessment
School of Civil and Environmental Engineering, Nanyang Technological University（新加坡南洋理工大学土木与环境工程学院）；Department of Business Administration, National Taiwan University（台湾国立台湾大学商业行政系）；School of Maritime Economics and Management, Dalian Maritime University（中国大连海事大学航海经济管理学院）；The Key Laboratory of Road and Traffic Engineering of the Ministry of Education, Tongji University（同济大学交通工程教育部长三角实验室）
Diversity Optimization for Travelling Salesman Problem via Deep Reinforcement Learning
South China University of Technology（华南理工大学）；Singapore Management University（新加坡管理大学）；Massachusetts Institute of Technology（麻省理工学院）；Eindhoven University of Technology（埃因霍温理工大学）
An End-to-End Deep Reinforcement Learning Approach for Solving the Traveling Salesman Problem with Drones
School of Management Science and Real Estate, Chongqing University（重庆大学管理科学与房地产学院）；School of Management, Guizhou University（贵州大学管理学院）；Chongqing Changan Minsheng APLL Logistics Co., Ltd.（重庆长安民生APLL物流有限公司）；State Grid Chongqing Electric Power Company Material Branch（国网重庆市电力公司物资分公司）
MTL-KD: Multi-Task Learning Via Knowledge Distillation for Generalizable Neural Vehicle Routing Solver
College of Computer Science and Software Engineering, Shenzhen University（深圳大学计算机科学与软件工程学院）；School of Automation and Intelligent Manufacturing, Southern University of Science and Technology（南方科技大学自动化与智能制造学院）；Guangdong Provincial Key Laboratory of Fully Actuated System Control Theory and Technology, Southern University of Science and Technology（广东省全驱动系统控制理论与技术重点实验室，南方科技大学）；Department of Industrial Engineering and Innovation Sciences, Eindhoven University of Technology（埃因霍温理工大学工业工程与创新科学系）
Deep Reinforcement Learning for Drone Route Optimization in Post-Disaster Road Assessment
School of Civil and Environmental Engineering, Nanyang Technological University（南洋理工大学土木与环境工程学院）；Department of Business Administration, National Taiwan University（台湾大学商业行政系）；School of Maritime Economics and Management, Dalian Maritime University（大连海事大学海运经济与管理学院）；The Key Laboratory of Road and Traffic Engineering of the Ministry of Education, Tongji University（同济大学交通工程教育部长期实验室）
The Hong Kong University of Science and Technology（香港科技大学）；University of Michigan（密歇根大学）；Didi Chuxing（滴滴出行）
With the rapid development of mobile-internet technologies, on-demand ride-sourcing services have become increasingly popular and largely reshaped the way people travel. Demand prediction is one of the most fundamental components in supply-demand management systems of ride-sourcing platforms. With accurate short-term prediction for origin-destination (OD) demand, the platforms make precise and timely decisions on real-time matching, idle vehicle reallocations and ride-sharing vehicle routing, etc. Compared to zone-based demand prediction that has been examined by many previous studies, OD-base…
University of Minnesota（明尼苏达大学）；University of Maryland（马里兰大学）
The eco-toll estimation problem quantifies the expected environmental cost (e.g., energy consumption, exhaust emissions) for a vehicle to travel along a path. This problem is important for societal applications such as eco-routing, which aims to find paths with the lowest exhaust emissions or energy need. The challenges of this problem are three-fold: (1) the dependence of a vehicle's eco-toll on its physical parameters; (2) the lack of access to data with eco-toll information; and (3) the influence of contextual information (i.e. the connections of adjacent segments in the path) on the eco-to…
