This paper introduces a reinforcement learning approach to optimize the Stochastic Vehicle Routing Problem with Time Windows (SVRP), focusing on reducing travel costs in goods delivery. We develop a novel SVRP formulation that accounts for uncertain travel costs and demands, alongside specific customer time windows. An attention-based neural network trained through reinforcement learning is employed to minimize routing costs. Our approach addresses a gap in SVRP research, which traditionally relies on heuristic methods, by leveraging machine learning. The model outperforms the Ant-Colony Optimization algorithm, achieving a 1.73% reduction in travel costs. It uniquely integrates external information, demonstrating robustness in diverse environments, making it a valuable benchmark for future SVRP studies and industry application.
In this paper we present an extension of the Nested Rollout Policy Adaptation algorithm (NRPA), namely the Generalized Nested Rollout Policy Adaptation (GNRPA), as well as its use for solving some instances of the Vehicle Routing Problem. We detail some results obtained on the Solomon instances set which is a conventional benchmark for the Vehicle Routing Problem (VRP). We show that on all instances, GNRPA performs better than NRPA. On some instances, it performs better than the Google OR Tool module dedicated to VRP.
Due to the practical importance of vehicle routing problems (VRP), there exists an ever-growing body of research in algorithms and (meta)heuristics for solving such problems. However, the diversity of VRP domains creates the separate problem of modeling such problems -- describing the domain entities (and, in particular, the planning decisions), the set of valid planning decisions, and the preferences between different plans. In this paper, we review the approaches for modeling vehicle routing problems. To make the comparison more straightforward, we formulate several criteria for evaluating m…
The Australian National University（澳大利亚国立大学）；Data61
In the classic Vehicle Routing Problem (VRP) a fleet of of vehicles has to visit a set of customers while minimising the operations' costs. We study a rich variant of the VRP featuring split deliveries, an heterogeneous fleet, and vehicle-commodity incompatibility constraints. Our goal is twofold: define the cheapest routing and the most adequate fleet. To do so, we split the problem into two interdependent components: a fleet design component and a routing component. First, we define two Mixed Integer Programming (MIP) formulations for each component. Then we discuss several improvements in t…
School of Technology, University of Campinas（坎皮纳斯大学技术学院）
The number of optimization techniques in the combinatorial domain is large and diversified. Nevertheless, there is still a lack of real benchmarks to validate optimization algorithms. In this work we introduce VRPBench, a tool to create instances and visualize solutions to the Vehicle Routing Problem (VRP) in a planar graph embedded in the Euclidean 2D space. We use VRPBench to model a real-world mail delivery case of the city of Artur Nogueira. Such scenarios were characterized as a multi-objective optimization of the VRP. We extracted a weighted graph from a digital map of the city to create…
The use of electric vehicles (EV) in the last mile is appealing from both sustainability and operational cost perspectives. In addition to the inherent cost efficiency of EVs, selling energy back to the grid during peak grid demand, is a potential source of additional revenue to a fleet operator. To achieve this, EVs have to be at specific locations (discharge points) during specific points in time (peak period), even while meeting their core purpose of delivering goods to customers. In this work, we consider the problem of EV routing with constraints on loading capacity; time window; vehicle-…
Huazhong University of Science and Technology（华中科技大学）；Huawei Noah’s Ark Lab（华为诺亚方舟实验室）
Vehicle routing is a well-known optimization research topic with significant practical importance. Among different approaches to solving vehicle routing, heuristics can produce a satisfactory solution at a reasonable computational cost. Consequently, much effort has been made in the past decades to develop vehicle routing heuristics. In this article, we systematically survey the existing vehicle routing heuristics, particularly on works carried out in recent years. A classification of vehicle routing heuristics is presented, followed by a review of their methodologies, recent developments, and…
TU Braunschweig（布伦瑞克工业大学）；University of Iowa（爱荷华大学）
There has been a paradigm-shift in urban logistic services in the last years; demand for real-time, instant mobility and delivery services grows. This poses new challenges to logistic service providers as the underlying stochastic dynamic vehicle routing problems (SDVRPs) require anticipatory real-time routing actions. Searching the combinatorial action space for efficient routing actions is by itself a complex task of mixed-integer programming (MIP) well-known by the operations research community. This complexity is now multiplied by the challenge of evaluating such actions with respect to th…
University of Auckland（奥克兰大学）；vWorkApp Inc.（vWorkApp公司）
In this thesis we present a new algorithm for the Vehicle Routing Problem called the Enhanced Bees Algorithm. It is adapted from a fairly recent algorithm, the Bees Algorithm, which was developed for continuous optimisation problems. We show that the results obtained by the Enhanced Bees Algorithm are competitive with the best meta-heuristics available for the Vehicle Routing Problem (within 0.5% of the optimal solution for common benchmark problems). We show that the algorithm has good runtime performance, producing results within 2% of the optimal solution within 60 seconds, making it suitab…
SNCF, the French public train company, is experimenting to develop new types of transportation services by tackling vehicle routing problems. While many deep learning models have been used to tackle efficiently vehicle routing problems, it is difficult to take into account time related constraints. In this paper, we solve the Capacitated Vehicle Routing Problem with Time Windows (CVRPTW) and the Capacitated Pickup and Delivery Problem with Time Windows (CPDPTW) with a constructive iterative Deep Learning algorithm. We use an Attention Encoder-Decoder structure and design a novel insertion heur…
We consider a wide family of vehicle routing problem variants with many complex and practical constraints, known as rich vehicle routing problems, which are faced on a daily basis by C.H. Robinson (CHR). Since CHR has many customers, each with distinct requirements, various routing problems with different objectives and constraints should be solved. We propose a set partitioning framework with a number of route generation algorithms, which have shown to be effective in solving a variety of different problems. The proposed algorithms have outperformed the existing technologies at CHR on 10 benc…
School of Technology, University of Campinas（坎皮纳斯大学技术学院）
The number of optimization techniques in the combinatorial domain is large and diversified. Nevertheless, real-world based benchmarks for testing algorithms are few. This work creates an extensible real-world mail delivery benchmark to the Vehicle Routing Problem (VRP) in a planar graph embedded in the 2D Euclidean space. Such problem is multi-objective on a roadmap with up to 25 vehicles and 30,000 deliveries per day. Each instance models one generic day of mail delivery, allowing both comparison and validation of optimization algorithms for routing problems. The benchmark may be extended to …
The Multi-Depot Vehicle Routing Problem (MDVRP) is a real-world model of the simplistic Vehicle Routing Problem (VRP) that considers how to satisfy multiple customer demands from numerous depots. This paper introduces a hybrid 2-stage approach based on two population-based algorithms - Ant Colony Optimization (ACO) that mimics ant behaviour in nature and the Imperialist Competitive Algorithm (ICA) that is based on geopolitical relationships between countries. In the proposed hybrid algorithm, ICA is responsible for customer assignment to the depots while ACO is routing and sequencing the custo…
Learning how to automatically solve optimization problems has the potential to provide the next big leap in optimization technology. The performance of automatically learned heuristics on routing problems has been steadily improving in recent years, but approaches based purely on machine learning are still outperformed by state-of-the-art optimization methods. To close this performance gap, we propose a novel large neighborhood search (LNS) framework for vehicle routing that integrates learned heuristics for generating new solutions. The learning mechanism is based on a deep neural network wit…
Nanyang Technological University（南洋理工大学）；Shandong University（山东大学）；National University of Singapore（新加坡国立大学）
We present a novel deep reinforcement learning method to learn construction heuristics for vehicle routing problems. In specific, we propose a Multi-Decoder Attention Model (MDAM) to train multiple diverse policies, which effectively increases the chance of finding good solutions compared with existing methods that train only one policy. A customized beam search strategy is designed to fully exploit the diversity of MDAM. In addition, we propose an Embedding Glimpse layer in MDAM based on the recursive nature of construction, which can improve the quality of each policy by providing more infor…
The Dynamic Vehicle Routing Problem with Time Windows (DVRPTW) is an extension of the well-known Vehicle Routing Problem (VRP), which takes into account the dynamic nature of the problem. This aspect requires the vehicle routes to be updated in an ongoing manner as new customer requests arrive in the system and must be incorporated into an evolving schedule during the working day. Besides the vehicle capacity constraint involved in the classical VRP, DVRPTW considers in addition time windows, which are able to better capture real-world situations. Despite this, so far, few studies have focused…
Wayne State University（韦恩州立大学）；U.S. Army CCDC Ground Vehicle Systems Center（美国陆军CCDC地面车辆系统中心）
An increase in greenhouse gases emission from the transportation sector has led companies and the government to elevate and support the production of electric vehicles (EV). With recent developments in urbanization and e-commerce, transportation companies are replacing their conventional fleet with EVs to strengthen the efforts for sustainable and environment-friendly operations. However, deploying a fleet of EVs asks for efficient routing and recharging strategies to alleviate their limited range and mitigate the battery degradation rate. In this work, a fleet of electric vehicles is consider…
Korea Advanced Institute of Science and Technology (KAIST)（韩国科学技术院（KAIST））
This paper introduces a new routing problem referred to as the vehicle routing problem with vector profits. Given a network composed of nodes (depot/sites) and arcs connecting the nodes, the problem determines routes that depart from the depot, visit sites to collect profits, and return to the depot. There are multiple stakeholders interested in the mission and each site is associated with a vector whose k-th element represents the profit value for the k-th stakeholder. The objective of the problem is to maximize the profit sum for the least satisfied stakeholder, i.e., the stakeholder with th…
Pontifícia Universidade Católica do Rio de Janeiro（里约热内卢天主教大学）；CIRRELT（企业网络、物流与运输跨大学研究中心）；HEC Montréal（蒙特利尔高等商学院）；University of Vienna（维也纳大学）
Vehicle routing problems have been the focus of extensive research over the past sixty years, driven by their economic importance and their theoretical interest. The diversity of applications has motivated the study of a myriad of problem variants with different attributes. In this article, we provide a concise overview of existing and emerging problem variants. Models are typically refined along three lines: considering more relevant objectives and performance metrics, integrating vehicle routing evaluations with other tactical decisions, and capturing fine-grained yet essential aspects of mo…
Universidad Adolfo Ibáñez（阿道夫·伊瓦涅斯大学）；Universidad Diego Portales（迭戈·波塔莱斯大学）；AGH University of Science and Technology（AGH科技大学）
This study introduces the Territory Design for Dynamic Multi-Period Vehicle Routing Problem with Time Windows (TD-DMPVRPTW), motivated by a real-world application at a food company's distribution center. This problem deals with the design of contiguous and compact territories for delivery of orders from a depot to a set of customers, with time windows, over a multi-period planning horizon. Customers and their demands vary dynamically over time. The problem is modeled as a mixed-integer linear program (MILP) and solved by a proposed heuristic. The heuristic solutions are compared with the propo…
University of Deusto（德乌斯托大学）；Ikerbasque, Basque Foundation for Science（伊克巴斯克，巴斯克科学基金会）；Universidad de Guadalajara（瓜达拉哈拉大学）
Electric vehicles (EVs) have been adopted in urban areas to reduce environmental pollution and global warming as a result of the increasing number of freight vehicles. However, there are still deficiencies in routing the trajectories of last-mile logistics that continue to impact social and economic sustainability. For that reason, in this paper, a hyper-heuristic (HH) approach called Hyper-heuristic Adaptive Simulated Annealing with Reinforcement Learning (HHASA$_{RL}$) is proposed. It is composed of a multi-armed bandit method and the self-adaptive Simulated Annealing (SA) metaheuristic algo…
Universidad Adolfo Ibáñez（阿道夫·伊瓦涅斯大学）；Universidad Diego Portales（迭戈·波塔莱斯大学）；AGH University of Science and Technology（AGH科技大学）
We develop a heuristic for the Consistent Vehicle Routing Problem with Time Windows (ConVRPTW), which is motivated by a real-world application at a food company's distribution center. Besides standard VRPTW restrictions, ConVRPTW assigns each customer just one driver to fulfill their orders during the whole multi-period planning horizon. For each driver and period, a route is sought to serve all their customers with positive demand. For each customer, the number of periods between consecutive orders and the ordered quantities are highly irregular. This causes difficulties in the daily routing,…
In real life, providing security for a set of large areas by covering the area with Unmanned Aerial Vehicles (UAVs) is a difficult problem that consist of multiple objectives. These difficulties are even greater if the area coverage must continue throughout a specific time window. We address this by considering a Vehicle Routing Problem with Time Windows (VRPTW) variation in which capacity of agents is one and each customer (target area) must be supplied with more than one vehicles simultaneously without violating time windows. In this problem, our aim is to find a way to cover all areas with …
Industry 4.0 is a concept that assists companies in developing a modern supply chain (MSC) system when they are faced with a dynamic process. Because Industry 4.0 focuses on mobility and real-time integration, it is a good framework for a dynamic vehicle routing problem (DVRP). This research works on DVRP. The aim of this research is to minimize transportation cost without exceeding the capacity constraint of each vehicle while serving customer demands from a common depot. Meanwhile, new orders arrive at a specific time into the system while the vehicles are executing the delivery of existing …
The Vehicle Routing Problem (VRP) is the combinatorial optimization problem of designing routes for vehicles to visit customers in such a fashion that a cost function, typically the number of vehicles, or the total travelled distance is minimized. The problem finds applications in industrial scenarios, for example where Automated Guided Vehicles run through the plant to deliver components from the warehouse. This specific problem, henceforth called the Electric Conflict-Free Vehicle Routing Problem (CF-EVRP), involves constraints such as limited operating range of the vehicles, time windows on…
In this paper fuzzy VRPTW with an uncertain travel time is considered. Credibility theory is used to model the problem and specifies a preference index at which it is desired that the travel times to reach the customers fall into their time windows. We propose the integration of fuzzy and ant colony system based evolutionary algorithm to solve the problem while preserving the constraints. Computational results for certain benchmark problems having short and long time horizons are presented to show the effectiveness of the algorithm. Comparison between different preferences indexes have been ob…
Middlesex University（密德萨斯大学）；Deusto Institute of Technology (DeustoTech)（德乌斯托技术学院（DeustoTech））；University of Deusto（德乌斯托大学）
A real-world newspaper distribution problem with recycling policy is tackled in this work. In order to meet all the complex restrictions contained in such a problem, it has been modeled as a rich vehicle routing problem, which can be more specifically considered as an asymmetric and clustered vehicle routing problem with simultaneous pickup and deliveries, variable costs and forbidden paths (AC-VRP-SPDVCFP). This is the first study of such a problem in the literature. For this reason, a benchmark composed by 15 instances has been also proposed. In the design of this benchmark, real geographica…
We investigate a learning decision support system for vehicle routing, where the routing engine learns implicit preferences that human planners have when manually creating route plans (or routings). The goal is to use these learned subjective preferences on top of the distance-based objective criterion in vehicle routing systems. This is an alternative to the practice of distinctively formulating a custom VRP for every company with its own routing requirements. Instead, we assume the presence of past vehicle routing solutions over similar sets of customers, and learn to make similar choices. T…
Multi-vehicle routing problem with soft time windows (MVRPSTW) is an indispensable constituent in urban logistics distribution systems. Over the past decade, numerous methods for MVRPSTW have been proposed, but most are based on heuristic rules that require a large amount of computation time. With the current rapid increase of logistics demands, traditional methods incur the dilemma between computational efficiency and solution quality. To efficiently solve the problem, we propose a novel reinforcement learning algorithm called the Multi-Agent Attention Model that can solve routing problem ins…
The goal of this paper is to investigate a decision support system for vehicle routing, where the routing engine learns from the subjective decisions that human planners have made in the past, rather than optimizing a distance-based objective criterion. This is an alternative to the practice of formulating a custom VRP for every company with its own routing requirements. Instead, we assume the presence of past vehicle routing solutions over similar sets of customers, and learn to make similar choices. The approach is based on the concept of learning a first-order Markov model, which correspond…
University of Maryland, College Park（马里兰大学帕克分校）；ESG UQAM（魁北克大学蒙特利尔分校管理科学学院）；GERAD（决策分析研究小组）；Verisk Computational and Human Intelligence Laboratory（Verisk计算与人类智能实验室）
We address the problem of accelerating column generation for set cover problems in which we relax the state space of the columns to do efficient pricing. We achieve this by adapting the recently introduced smooth and flexible dual optimal inequalities (DOI) for use with relaxed columns. Smooth DOI exploit the observation that similar items are nearly fungible, and hence should be associated with similarly valued dual variables. Flexible DOI exploit the observation that the change in cost of a column induced by removing an item can be bounded. We adapt these DOI to the problem of capacitated ve…
We formulate and solve the H-SARA Problem, a Vehicle Routing and Appointment Scheduling Problem motivated by home services management. We assume that travel times, service durations, and customer cancellations are stochastic. We use a two-stage process that first generates teams and routes using a VRP Solver with optional extensions and then uses an MC Scheduler that determines expected arrival times by teams at customers. We further introduce two different models of cancellation and their associated impacts on routing and scheduling. Finally, we introduce the Route Fracture Metaheuristic that…
The University of Adelaide（阿德莱德大学）；University of Münster（明斯特大学）
In practice, e.g. in delivery and service scenarios, Vehicle-Routing-Problems (VRPs) often imply repeated decision making on dynamic customer requests. As in classical VRPs, tours have to be planned short while the number of serviced customers has to be maximized at the same time resulting in a multi-objective problem. Beyond that, however, dynamic requests lead to the need for re-planning of not yet realized tour parts, while already realized tour parts are irreversible. In this paper we study this type of bi-objective dynamic VRP including sequential decision making and concurrent realizatio…
Laboratory of Innovations in Transportation (LiTrans)（交通创新实验室）；Ryerson University（瑞尔森大学）
This study exploits the advancements in information and communication technology (ICT), connected and automated vehicles (CAVs), and sensing, to develop proactive multi-objective eco-routing strategies. For a robust application, several GHG costing approaches are examined. The predictive models for the link level traffic and emission states are developed using long short term memory deep network with exogenous predictors. It is found that proactive routing strategies outperformed the myopic strategies, regardless of the routing objective. Whether myopic or proactive, the multi-objective routin…
Traffic congestion in urban road networks leads to longer trip times and higher emissions, especially during peak periods. While the Shortest Path First (SPF) algorithm is optimal for a single vehicle in a static network, it performs poorly in dynamic, multi-vehicle settings, often worsening congestion by routing all vehicles along identical paths. We address dynamic vehicle routing through a multi-agent reinforcement learning (MARL) framework for coordinated, network-aware fleet navigation. We first propose Adaptive Navigation (AN), a decentralized MARL model where each intersection agent pro…
This paper develops an inherently parallelised, fast, approximate learning-based solution to the generic class of Capacitated Vehicle Routing Problems with Time Windows and Dynamic Routing (CVRP-TWDR). Considering vehicles in a fleet as decentralised agents, we postulate that using reinforcement learning (RL) based adaptation is a key enabler for real-time route formation in a dynamic environment. The methodology allows each agent (vehicle) to independently evaluate the value of serving each customer, and uses a centralised allocation heuristic to finalise the allocations based on the generate…
In horizontal collaborations, carriers form coalitions in order to perform parts of their logistics operations jointly. By exchanging transportation requests among each other, they can operate more efficiently and in a more sustainable way. Collaborative vehicle routing has been extensively discussed in the literature. We identify three major streams of research: (i) centralized collaborative planning, (ii) decentralized planning without auctions, and (ii) auction-based decentralized planning. For each of them we give a structured overview on the state of knowledge and discuss future research …
University of Toronto（多伦多大学）；Western University（韦仕敦大学）；University of Waterloo（滑铁卢大学）
The past decade has seen a rapid penetration of electric vehicles (EV) in the market, more and more logistics and transportation companies start to deploy EVs for service provision. In order to model the operations of a commercial EV fleet, we utilize the EV routing problem with time windows (EVRPTW). In this research, we propose an end-to-end deep reinforcement learning framework to solve the EVRPTW. In particular, we develop an attention model incorporating the pointer network and a graph embedding technique to parameterize a stochastic policy for solving the EVRPTW. The model is then traine…
