利用分层强化学习优化多集群疫情控制的资源受限非药物干预
Optimizing Resource-Constrained Non-Pharmaceutical Interventions for Multi-Cluster Outbreak Control Using Hierarchical Reinforcement Learning
AI总结:
本文提出分层强化学习框架,用于在资源受限条件下优化多集群疫情控制的非药物干预策略,通过动态模拟SARS-CoV-2疫情,验证了该方法在提升疫情控制效果和可扩展性方面的优势。
摘要:
非药物干预(NPIs),如诊断检测和隔离,对于控制传染病爆发至关重要,但常受资源限制,特别是在爆发早期。在现实公共卫生环境中,资源必须在多个异步出现、规模和风险各异的爆发集群之间分配,并竞争共享的资源预算。本文将该问题建模为受约束的 restless 多臂老虎机,并提出分层强化学习框架。全局控制器学习连续动作成本乘数以调整全局资源需求,而广义局部策略估计在每个集群内分配资源给个体的边际价值。在现实的基于代理的 SARS-CoV-2 模拟器中评估了所提出的框架。在广泛的系统规模和测试预算范围内,我们的方法在 RMAB 激发和启发式基线方法上表现更优,将疫情控制效果提高了 20%-30%。在最多 40 个同时活跃的集群上进行的实验进一步证明,分层框架具有高度可扩展性,并比 RMAB 激发方法实现更快的决策速度。
英文摘要:
Non-pharmaceutical interventions (NPIs), such as diagnostic testing and quarantine, are crucial for controlling infectious disease outbreaks but are often constrained by limited resources, particularly in early outbreak stages. In real-world public health settings, resources must be allocated across multiple outbreak clusters that emerge asynchronously, vary in size and risk, and compete for a shared resource budget. Here, a cluster corresponds to a group of close contacts generated by a single infected index case. Thus, decisions must be made under uncertainty and heterogeneous demands, while respecting operational constraints. We formulate this problem as a constrained restless multi-armed bandit and propose a hierarchical reinforcement learning framework. A global controller learns a continuous action cost multiplier that adjusts global resource demand, while a generalized local policy estimates the marginal value of allocating resources to individuals within each cluster. We evaluate the proposed framework in a realistic agent-based simulator of SARS-CoV-2 with dynamically arriving clusters. Across a wide range of system scales and testing budgets, our method consistently outperforms RMAB-inspired and heuristic baselines, improving outbreak control effectiveness by 20%-30%. Experiments on up to 40 concurrently active clusters further demonstrate that the hierarchical framework is highly scalable and enables faster decision-making than the RMAB-inspired method.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。