arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 1905.04735stat.ME
收藏

大型决策问题的Thompson抽样

Note on Thompson sampling for large decision problems

Tao Hu, Eric B. Laber, Zhen Li, Nick J. Meyer, Krishna Pacifici

摘要:

越来越多的人对使用流数据来为各种应用领域的决策提供信息感兴趣,包括移动健康,食品安全,安全和资源管理。决策支持系统将在线决策形式化为从最新信息到推荐决策的映射。从流数据在线估计最优决策策略需要同时估计底层系统动态的组件以及给定这些动态的最优决策策略;因此,在选择导致改进估计的决策和选择基于当前估计似乎是最优的决策之间存在固有的权衡。汤普森(Thompson,1933)是第一个在为一系列患者选择两种治疗方法的背景下将这种权衡正式化的人之一;他提出了一种简单的启发式方法,即在每个时间点随机选择一种治疗方法,选择概率与其最优的后验概率成正比。我们认为汤普森采样的一个变种,是简单的实现,并可以应用于大型和复杂的决策问题。我们表明,建议汤普森抽样估计是一致的最佳决策支持系统,并提供收敛速度和有限样本误差界。所提出的算法说明了使用基于代理的模型的网络和管理的绿头鸭种群的流感传播。

英文摘要:

There is increasing interest in using streaming data to inform decision making across a wide range of application domains including mobile health, food safety, security, and resource management. A decision support system formalizes online decision making as a map from up-to-date information to a recommended decision. Online estimation of an optimal decision strategy from streaming data requires simultaneous estimation of components of the underlying system dynamics as well as the optimal decision strategy given these dynamics; thus, there is an inherent trade-off between choosing decisions that lead to improved estimates and choosing decisions that appear to be optimal based on current estimates. Thompson (1933) was among the first to formalize this trade-off in the context of choosing between two treatments for a stream of patients; he proposed a simple heuristic wherein a treatment is selected randomly at each time point with selection probability proportional to the posterior probability that it is optimal. We consider a variant of Thompson sampling that is simple to implement and can be applied to large and complex decision problems. We show that the proposed Thompson sampling estimator is consistent for the optimal decision support system and provide rates of convergence and finite sample error bounds. The proposed algorithm is illustrated using an agent-based model of the spread of influenza on a network and management of mallard populations in the United States.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑