We introduce a combinatorial optimization-enriched machine learning pipeline and a novel learning paradigm to solve inventory routing problems with stochastic demand and dynamic inventory updates. After each inventory update, our approach reduces replenishment and routing decisions to an optimal solution of a capacitated prize-collecting traveling salesman problem for which well-established algorithms exist. Discovering good prize parametrizations is non-trivial; therefore, we have developed a machine learning approach. We evaluate the performance of our pipeline in settings with steady-state and more complex demand patterns. Compared to previous works, the policy generated by our algorithm leads to significant cost savings, achieves lower inference time, and can even leverage contextual information.
In this work we compare several new computational approaches to an inventory routing problem, in which a single product is shipped from a warehouse to retailers via an uncapacitated vehicle. We survey exact algorithms for the Traveling Salesman Problem (TSP) and its relaxations in the literature for the routing component. For the inventory control component we survey classical mixed integer linear programming and shortest path formulations for inventory models. We present a numerical study comparing combinations of the two empirically in terms of cost and solution time.
Recent developments in urbanization and e-commerce have pushed businesses to deploy efficient systems to decrease their supply chain cost. Vendor Managed Inventory (VMI) is one of the most widely used strategies to effectively manage supply chains with multiple parties. VMI implementation asks for solving the Inventory Routing Problem (IRP). This study considers a multi-product multi-period inventory routing problem, including a supplier, set of customers, and a fleet of heterogeneous vehicles. Due to the complex nature of the IRP, we developed a Modified Adaptive Genetic Algorithm (MAGA) to s…
The two-echelon inventory-routing problem (2E-IRP) addresses the coordination of inventory management and freight transportation throughout a two-echelon supply network. The latter consists of geographically widespread customers whose demand over a discrete planning horizon can be met from either their local inventory or intermediate facilities' inventory. Intermediate facilities are located in the city outskirt and are supplied from distant suppliers. The 2E-IRP aims to minimize transportation costs and inventory costs while meeting customers' demand. A route-based formulation is proposed and…
Given the rapid advances in unmanned aerial vehicles, or drones, and increasing need to monitor traffic at a city level, one of the current research gaps is how to systematically deploy drones over multiple periods. We propose a real-time data-driven approach: we formulate the first deterministic arc-inventory routing problem and derive its stochastic dynamic policy. The policy is expected to be of greatest value in scenarios where uncertainty is highest and costliest, such as city traffic monitoring during major events. The Bellman equation for an approximation of the proposed inventory routi…
Tecnologico de Monterrey（蒙特雷科技大学）；Universidade Federal da Bahia（巴伊亚联邦大学）
We consider an NP-hard selective and periodic inventory routing problem (SPIRP) in a waste vegetable oil collection environment. This SPIRP arises in the context of reverse logistics where a biodiesel company has daily requirements of oil to be used as raw material in its production process. These requirements can be fulfilled by using the available inventory, collecting waste vegetable oil or purchasing virgin oil. The problem consists in determining a period (cyclic) planning for the collection and purchasing of oil such that the total collection, inventory and purchasing costs are minimized…
We introduce an algorithm design technique for a class of combinatorial optimization problems with concave costs. This technique yields a strongly polynomial primal-dual algorithm for a concave cost problem whenever such an algorithm exists for the fixed-charge counterpart of the problem. For many practical concave cost problems, the fixed-charge counterpart is a well-studied combinatorial optimization problem. Our technique preserves constant factor approximation ratios, as well as ratios that depend only on certain problem parameters, and exact algorithms yield exact algorithms. Using our te…
The multi Vehicle Routing Problem with Pickup and Delivery with Time Windows is a challenging version of the Vehicle Routing Problem. In this paper, by embedding many complex assignment routing constraints through constructing a multi dimensional network, we intend to reach optimality for local clusters derived from a reasonably large set of passengers on real world transportation networks. More specifically, we introduce a multi vehicle state space time network representation in which only the non dominated assignment based hyper paths are examined. In addition, by the aid of passengers cumul…
MIT Operations Research Center（麻省理工学院运筹学中心）；Google Research（谷歌研究院）
Value-function-based methods have long played an important role in reinforcement learning. However, finding the best next action given a value function of arbitrary complexity is nontrivial when the action space is too large for enumeration. We develop a framework for value-function-based deep reinforcement learning with a combinatorial action space, in which the action selection problem is explicitly formulated as a mixed-integer optimization problem. As a motivating example, we present an application of this framework to the capacitated vehicle routing problem (CVRP), a combinatorial optimiz…
Disjunctive Submodular Functions: Envelopes and Applications to Inventory and 0-1 Quadratic Optimization
Antai College of Economics and Management, Shanghai Jiao Tong University（上海交通大学安泰经济与管理学院）；Mitch Daniels School of Business, Purdue University（普渡大学米切尔·丹尼尔斯商学院）
Recently, the applications of the methodologies of Reinforcement Learning (RL) to NP-Hard Combinatorial optimization problems have become a popular topic. This is essentially due to the nature of the traditional combinatorial algorithms, often based on a trial-and-error process. RL aims at automating this process. At this regard, this paper focuses on the application of RL for the Vehicle Routing Problem (VRP), a famous combinatorial problem that belongs to the class of NP-Hard problems. In this work, first, the problem is modeled as a Markov Decision Process (MDP) and then the PPO method (whi…
Universitat Pompeu Fabra（庞培法布拉大学）；Barcelona Graduate School of Economics（巴塞罗那经济研究生院）；Barcelona School of Management（巴塞罗那管理学院）
We consider a repeated newsvendor problem where the inventory manager has no prior information about the demand, and can access only censored/sales data. In analogy to multi-armed bandit problems, the manager needs to simultaneously "explore" and "exploit" with her inventory decisions, in order to minimize the cumulative cost. We make no probabilistic assumptions---importantly, independence or time stationarity---regarding the mechanism that creates the demand sequence. Our goal is to shed light on the hardness of the problem, and to develop policies that perform well with respect to the regre…
Unmanned ground vehicles (UGVs) are being used extensively in civilian and military applications for applications such as underground mining, nuclear plant operations, planetary exploration, intelligence, surveillance and reconnaissance (ISR) missions and manned-unmanned teaming. We consider a multi-objective, multiple-vehicle routing problem in which teams of manned ground vehicles (MGVs) and UGVs are deployed respectively in a leader-follower framework to execute missions with differing requirements for MGVs and UGVs while considering human-robot interactions (HRI). HRI studies highlight the…
Bosch Center for AI（博世人工智能中心）；Technion – Israel Institute of Technology（以色列理工学院）
We study combinatorial problems with real world applications such as machine scheduling, routing, and assignment. We propose a method that combines Reinforcement Learning (RL) and planning. This method can equally be applied to both the offline, as well as online, variants of the combinatorial problem, in which the problem components (e.g., jobs in scheduling problems) are not known in advance, but rather arrive during the decision-making process. Our solution is quite generic, scalable, and leverages distributional knowledge of the problem parameters. We frame the solution process as an MDP, …
Single- vs. Dual-Policy Reinforcement Learning for Dynamic Bike Rebalancing
Department of Mathematical and Industrial Engineering, Polytechnique Montréal（蒙特利尔理工学院数学与工业工程系）；School of Management, Université du Québec à Montréal（魁北克大学蒙特利尔分校管理学院）；Cornell Tech and Technion - IIT（康奈尔科技与技术Ion研究所）
We investigate optimal routing and scheduling strategies for multi-hop wireless networks with rateless codes. Rateless codes allow each node of the network to accumulate mutual information from every packet transmission. This enables a significant performance gain over conventional shortest path routing. Further, it outperforms cooperative communication techniques that are based on energy accumulation. However, it requires complex and combinatorial networking decisions concerning which nodes participate in transmission, and which decode ordering to use. We formulate three problems of interest …
Many emerging commercial services are based on the sharing or pooling of resources for common use with the aim of reducing costs. Businesses such as delivery-, mobility-, or transport-as-a-service have become standard in many parts of the world, fulfilling on-demand requests for customers in live settings. However, it is known that many of these problems are NP-hard, and therefore both modeling and solving them accurately is a challenge. Here we focus on one such routing problem, the Ride Pooling Problem (RPP), where multiple customers can request on-demand pickups and drop-offs from shared ve…
In the last years, there has been a great interest in machine-learning-based heuristics for solving NP-hard combinatorial optimization problems. The developed methods have shown potential on many optimization problems. In this paper, we present a learned heuristic for the reoptimization of a problem after a minor change in its data. We focus on the case of the capacited vehicle routing problem with static clients (i.e., same client locations) and changed demands. Given the edges of an original solution, the goal is to predict and fix the ones that have a high chance of remaining in an optimal …
Learning for routing: A guided review of recent developments and future directions
Architecture and Civil Engineering, Chalmers University of Technology（建筑与土木工程，楚姆勒斯技术大学）；Electrical Engineering, Chalmers University of Technology（电气工程，楚姆勒斯技术大学）；Computer Science and Engineering, Chalmers University of Technology and University of Gothenburg（计算机科学与工程，楚姆勒斯技术大学和哥德堡大学）；Management, University of Bath（管理，巴斯大学）
Quantum annealing technologies aim to solve computational optimization and sampling problems. QPU (Quantum Processing Unit) machines such as the D-Wave system use the QUBO (Quadratic Unconstrained Binary Optimization) formula to define model optimization problems for quantum annealing. This machine uses quantum effects to speed up computing time better than classical computers. We propose a vehicle routing problem that can be formulated in the QUBO model as a combinatorial problem, which gives the possible route solutions increases exponentially. The solution aims to optimize the vehicle's jou…
We consider the problem of vehicle routing for Autonomous Mobility-on-Demand (AMoD) systems, wherein a fleet of self-driving vehicles provides on-demand mobility in a given environment. Specifically, the task it to compute routes for the vehicles (both customer-carrying and empty travelling) so that travel demand is fulfilled and operational cost is minimized. The routing process must account for congestion effects affecting travel times, as modeled via a volume-delay function (VDF). Route planning with VDF constraints is notoriously challenging, as such constraints compound the combinatorial …
Combinatorial optimization is a well-established area in operations research and computer science. Until recently, its methods have focused on solving problem instances in isolation, ignoring that they often stem from related data distributions in practice. However, recent years have seen a surge of interest in using machine learning as a new approach for solving combinatorial problems, either directly as solvers or by enhancing exact solvers. Based on this context, the ML4CO aims at improving state-of-the-art combinatorial optimization solvers by replacing key heuristic components. The compet…
Quantum annealing (QA) is a quantum computing algorithm that works on the principle of Adiabatic Quantum Computation (AQC), and it has shown significant computational advantages in solving combinatorial optimization problems such as vehicle routing problems (VRP) when compared to classical algorithms. This paper presents a QA approach for solving a variant VRP known as multi-depot capacitated vehicle routing problem (MDCVRP). This is an NP-hard optimization problem with real-world applications in the fields of transportation, logistics, and supply chain management. We consider heterogeneous de…
University of Southern California（南加州大学）；University of Michigan（密歇根大学）
Combinatorial network optimization algorithms that compute optimal structures taking into account edge weights form the foundation for many network protocols. Examples include shortest path routing, minimal spanning tree computation, maximum weighted matching on bipartite graphs, etc. We present CLRMR, the first online learning algorithm that efficiently solves the stochastic version of these problems where the underlying edge weights vary as independent Markov chains with unknown dynamics. The performance of an online learning algorithm is characterized in terms of regret, defined as the cumu…
With the continued growth of Internet traffic, and the scarcity of the optical spectrum, there is a continuous need to optimize the usage of this resource. In the process of provisioning elastic optical networks using the flexible frequency grid paradigm, telecommunication operators must deal with a combinatorial optimization problem that is NP-complete namely the Routing and Spectrum Allocation(RSA) problem. Following on our previous study, where we used Integer Linear Programming, and proposed a Column Generation algorithm based on a Lightpath decomposition, which proved to be the most effic…
