Pareto Bandits下无信息先验Thompson抽样的最优性
Optimality of Thompson Sampling with Noninformative Priors for Pareto Bandits
摘要：
在随机多臂强盗问题中，一种称为Thompson抽样（TS）的随机概率匹配策略在各种奖励模型中表现出了优异的性能。除了经验性能，TS已被证明在几个模型中实现渐近问题相关的下限。然而，它的最优性主要是在属于指数族的轻尾或单参数模型下解决的。在本文中，我们考虑TS的最优性的Pareto模型，有一个沉重的尾巴，并由两个未知参数的参数。具体来说，我们讨论了TS与概率匹配先验，包括Jeffreys先验和参考先验的最优性。我们首先证明了具有一定概率匹配先验的TS可以达到最优后悔界。然后，我们展示了TS与其他先验，包括Jeffreys和参考先验的次优性。然而，我们发现，TS与Jeffreys和参考先验可以实现渐近下界，如果使用截断过程。这些结果表明，仔细选择noninformative先验，以避免次优和TS为基础的政策截断程序的有效性。
英文摘要：
In the stochastic multi-armed bandit problem, a randomized probability matching policy called Thompson sampling (TS) has shown excellent performance in various reward models. In addition to the empirical performance, TS has been shown to achieve asymptotic problem-dependent lower bounds in several models. However, its optimality has been mainly addressed under light-tailed or one-parameter models that belong to exponential families. In this paper, we consider the optimality of TS for the Pareto model that has a heavy tail and is parameterized by two unknown parameters. Specifically, we discuss the optimality of TS with probability matching priors that include the Jeffreys prior and the reference priors. We first prove that TS with certain probability matching priors can achieve the optimal regret bound. Then, we show the suboptimality of TS with other priors, including the Jeffreys and the reference priors. Nevertheless, we find that TS with the Jeffreys and reference priors can achieve the asymptotic lower bound if one uses a truncation procedure. These results suggest carefully choosing noninformative priors to avoid suboptimality and show the effectiveness of truncation procedures in TS-based policies.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
