We study a general problem of allocating limited resources to heterogeneous customers over time under model uncertainty. Each type of customer can be serviced using different actions, each of which stochastically consumes some combination of resources, and returns different rewards for the resources consumed. We consider a general model where the resource consumption distribution associated with each (customer type, action)-combination is not known, but is consistent and can be learned over time. In addition, the sequence of customer types to arrive over time is arbitrary and completely unknown.
We overcome both the challenges of model uncertainty and customer heterogeneity by judiciously synthesizing two algorithmic frameworks from the literature: inventory balancing, which "reserves" a portion of each resource for high-reward customer types which could later arrive, and online learning, which shows how to "explore" the resource consumption distributions of each customer type under different actions. We define an auxiliary problem, which allows for existing competitive ratio and regret bounds to be seamlessly integrated. Furthermore, we show that the performance guarantee generated by our framework is tight, that is, we provide an information-theoretic lower bound which shows that both the loss from competitive ratio and the loss for regret are relevant in the combined problem.
Finally, we demonstrate the efficacy of our algorithms on a publicly available hotel data set. Our framework is highly practical in that it requires no historical data (no fitted customer choice models, nor forecasting of customer arrival patterns) and can be used to initialize allocation strategies in fast-changing environments.
InvEvolve: Evolving White-Box Inventory Policies via Large Language Models with Performance Guarantees
Shanghai University of Finance and Economics(上海财经大学);Shanghai Jiao Tong University(上海交通大学);The Chinese University of Hong Kong, Shenzhen(香港中文大学(深圳));Tsinghua University(清华大学);Boston College(波士顿大学)
Online Learning of Weakly Coupled MDP Policies for Load Balancing and Auto Scaling
Indian Institute of Science(印度科学学院);IMT Atlantique Bretagne-Pays de la Loire(大西洋理工学院布列塔尼-卢瓦尔河地区校区);Institute of Computing, Federal University of Rio de Janeiro (UFRJ)(里约热内卢联邦大学计算机研究所)