arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2411.00515cs.LG
收藏

库存管理中的零样本泛化:训练,然后估计和决策

Zero-shot Generalization in Inventory Management: Train, then Estimate and Decide

Department of Industrial Engineering and Innovation Sciences, Eindhoven University of Technology(工业工程与创新科学系,埃因霍温理工大学)

Tarkan Temizöz, Christina Imdahl, Remco Dijkman, Douniel Lamghari-Idrissi, Willem van Jaarsveld

AI总结:

本文提出了一种针对库存管理的零样本泛化框架,通过训练一般能力代理以应对未知参数的决策问题,提升了在动态环境中的性能。

摘要:

在现实世界中部署深度强化学习(DRL)进行库存管理面临挑战,包括动态环境和不确定的问题参数,例如需求和交货时间分布。这些挑战突显出研究的空白,表明需要一个统一的框架来建模和解决在参数不确定性下的序列决策问题。我们通过探索DRL在库存管理中被忽视的领域来解决这一问题:在零样本泛化(ZSG)下训练一般能力代理(GCAs)。这里,GCAs是高级DRL策略,旨在处理具有多样库存挑战的广泛采样问题实例。ZSG指的是能够成功将学习的策略应用于未知参数的未见实例,而无需重新训练。我们提出了一种统一的超马尔可夫决策过程公式和训练,然后估计和决策(TED)框架,以训练和部署针对库存管理应用的GCA。TED框架由三个阶段组成:在各种问题实例上训练GCA,部署期间持续估计问题参数,并基于这些估计做出决策。应用于具有销售损失的周期性审查库存问题、循环需求模式和随机交货时间,我们训练的代理,即一般能力销售损失网络(GC-LSN),在问题参数已知时,始终优于传统策略。此外,在需求和/或交货时间分布最初未知且必须估计的情况下,我们与在线学习方法进行基准测试,这些方法提供最坏情况性能保证。我们的GC-LSN策略,与Kaplan-Meier估计器结合,被证明能够补充这些方法,通过提供更优越的经验性能。

英文摘要:

Deploying deep reinforcement learning (DRL) in real-world inventory management presents challenges, including dynamic environments and uncertain problem parameters, e.g. demand and lead time distributions. These challenges highlight a research gap, suggesting a need for a unifying framework to model and solve sequential decision-making under parameter uncertainty. We address this by exploring an underexplored area of DRL for inventory management: training generally capable agents (GCAs) under zero-shot generalization (ZSG). Here, GCAs are advanced DRL policies designed to handle a broad range of sampled problem instances with diverse inventory challenges. ZSG refers to the ability to successfully apply learned policies to unseen instances with unknown parameters without retraining. We propose a unifying Super-Markov Decision Process formulation and the Train, then Estimate and Decide (TED) framework to train and deploy a GCA tailored to inventory management applications. The TED framework consists of three phases: training a GCA on varied problem instances, continuously estimating problem parameters during deployment, and making decisions based on these estimates. Applied to periodic review inventory problems with lost sales, cyclic demand patterns, and stochastic lead times, our trained agent, the Generally Capable Lost Sales Network (GC-LSN) consistently outperforms well-known traditional policies when problem parameters are known. Moreover, under conditions where demand and/or lead time distributions are initially unknown and must be estimated, we benchmark against online learning methods that provide worst-case performance guarantees. Our GC-LSN policy, paired with the Kaplan-Meier estimator, is demonstrated to complement these methods by providing superior empirical performance.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑