Deploying deep reinforcement learning (DRL) in real-world inventory management presents challenges, including dynamic environments and uncertain problem parameters, e.g. demand and lead time distributions. These challenges highlight a research gap, suggesting a need for a unifying framework to model and solve sequential decision-making under parameter uncertainty. We address this by exploring an underexplored area of DRL for inventory management: training generally capable agents (GCAs) under zero-shot generalization (ZSG). Here, GCAs are advanced DRL policies designed to handle a broad range of sampled problem instances with diverse inventory challenges. ZSG refers to the ability to successfully apply learned policies to unseen instances with unknown parameters without retraining.
We propose a unifying Super-Markov Decision Process formulation and the Train, then Estimate and Decide (TED) framework to train and deploy a GCA tailored to inventory management applications. The TED framework consists of three phases: training a GCA on varied problem instances, continuously estimating problem parameters during deployment, and making decisions based on these estimates. Applied to periodic review inventory problems with lost sales, cyclic demand patterns, and stochastic lead times, our trained agent, the Generally Capable Lost Sales Network (GC-LSN) consistently outperforms well-known traditional policies when problem parameters are known. Moreover, under conditions where demand and/or lead time distributions are initially unknown and must be estimated, we benchmark against online learning methods that provide worst-case performance guarantees. Our GC-LSN policy, paired with the Kaplan-Meier estimator, is demonstrated to complement these methods by providing superior empirical performance.
DeepStock: Reinforcement Learning with Policy Regularizations for Inventory Management
Booth School of Business, University of Chicago(芝加哥大学商学院);Taobao & Tmall Group(淘宝与天猫集团);School of Economics, Sichuan University(四川大学经济学院);Graduate School of Business, Columbia University(哥伦比亚大学商学院);School of Operations Research and Information Engineering, Cornell University(康奈尔大学运筹学与信息工程学院)
Multi-Agent Reinforcement Learning with Shared Resources for Inventory Management
Huazhong University of Science and Technology(华中科技大学);University of Science and Technology of China(中国科学技术大学);Carnegie Mellon University(卡内基梅隆大学);University of Illinois at Urbana-Champaign(伊利诺伊大学厄巴纳-香槟分校);Microsoft Research(微软研究院)
ORPR: An OR-Guided Pretrain-then-Reinforce Learning Model for Inventory Management
Department of Industrial Engineering, Tsinghua University(清华大学工业工程系);Supply Chain Tech Team Y, JD.com(京东供应链技术团队);Faculty of Engineering and Faculty of Business and Economics, The University of Hong Kong(香港大学工程学院和商学院)
Deep Reinforcement Learning for Solving Management Problems: Towards A Large Management Mode
Random Seed Inc.(随机种子公司);Wuhan Institute for Artificial Intelligence(武汉人工智能研究院);Guanghua School of Management, Peking University(北京大学光华管理学院);Xiangjiang Laboratory(湘江实验室);H. Milton Stewart School of Industrial and Systems Engineering, Georgia Institute of Technology(佐治亚理工学院H·米尔顿·斯图尔特工业与系统工程学院);Institute for Artificial Intelligence, Peking University(北京大学人工智能研究院)
QAmplifyNet: Pushing the Boundaries of Supply Chain Backorder Prediction Using Interpretable Hybrid Quantum-Classical Neural Network
Khulna University of Engineering and Technology (KUET)(库尔纳工程技术大学(KUET));American International University-Bangladesh (AIUB)(孟加拉国美国国际大学(AIUB));The University of Aizu(会津大学)
Control of Dual-Sourcing Inventory Systems using Recurrent Neural Networks
Frankfurt School of Finance and Management(法兰克福金融管理学院);University of California, Los Angeles(加利福尼亚大学洛杉矶分校);University of Zurich(苏黎世大学);Erasmus University Rotterdam(鹿特丹伊拉斯姆斯大学)
Predictive and Prescriptive Performance of Bike-Sharing Demand Forecasts for Inventory Management
Technical University of Denmark(丹麦技术大学);Technical University of Munich(慕尼黑工业大学);University of Groningen(格罗宁根大学);University of Twente(特文特大学);Munich Data Science Institute(慕尼黑数据科学研究所)
Data-driven inventory management for new products: An adjusted Dyna-$Q$ approach with transfer learning
Department of Data and Systems Engineering, The University of Hong Kong(数据与系统工程系,香港大学);Musketeers Foundation Institute of Data Science, The University of Hong Kong(穆斯克特基金会数据科学研究所,香港大学)
Booth School of Business, University of Chicago(芝加哥大学商学院);Graduate School of Business and Data Science Institute, Columbia University(哥伦比亚大学研究生商学院和数据科学研究院);School of Operations Research and Information Engineering, Cornell University(康奈尔大学运筹学与信息工程学院)