管理公共卫生配药点的结构化演员-评论家
Structured Actor-Critic for Managing Public Health Points-of-Dispensing
摘要：
公共卫生组织面临着分配治疗的问题（即，疫苗、抗生素和其他）在紧急情况期间通过“分发点”（POD）向受影响的人群群体分发，通常是在存在复杂性的情况下，如需求随机性、异质效用（例如，对于疫苗分配，可能需要优先考虑人口的某些部分）和有限的储存。我们制定了一个分层的马尔可夫决策过程（MDP）模型与两个层次的决策（和决策者）：上层决策来自一个库存计划，“控制”一个较低层次的动态问题，优化分配决策，考虑到异构效用函数的随机集的POD。然后，我们推导出的MDP模型的结构特性，并提出了一个近似的动态规划（ADP）算法，利用结构的政策和价值空间（状态依赖的基本库存和库存，分别）。该算法可以被认为是一个演员-评论家的方法，据我们所知，本文是第一个共同利用政策和价值结构内的演员-评论家的框架。我们证明，政策和价值函数近似每个收敛到他们的最佳对手的概率为1，并提供了一个全面的数值分析显示改进的经验收敛速度相比，其他ADP技术。最后，我们展示了如何将我们算法的基于聚合的版本应用于现实案例研究中，用于在持续的阿片类药物危机中通过第一反应者分配纳洛酮（一种过量逆转药物）的问题。
英文摘要：
Public health organizations face the problem of dispensing treatments (i.e., vaccines, antibiotics, and others) to groups of affected populations through "points-of-dispensing" (PODs) during emergency situations, typically in the presence of complexities like demand stochasticity, heterogenous utilities (e.g., for vaccine distribution, certain segments of the population may need to be prioritized), and limited storage. We formulate a hierarchical Markov decision process (MDP) model with two levels of decisions (and decision-makers): the upper-level decisions come from an inventory planner that "controls" a lower-level dynamic problem, which optimizes dispensing decisions that take into consideration the heterogeneous utility functions of the random set of PODs. We then derive structural properties of the MDP model and propose an approximate dynamic programming (ADP) algorithm that leverages structure in both the policy and the value space (state-dependent basestocks and concavity, respectively). The algorithm can be considered an actor-critic method; to our knowledge, this paper is the first to jointly exploit policy and value structure within an actor-critic framework. We prove that the policy and value function approximations each converge to their optimal counterparts with probability one and provide a comprehensive numerical analysis showing improved empirical convergence rates when compared to other ADP techniques. Finally, we show how an aggregation-based version of our algorithm can be applied in a realistic case study for the problem of dispensing naloxone (an overdose reversal drug) via first responders amidst the ongoing opioid crisis.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
