分类学习的比率指数及其应用
The Ratio Index for Budgeted Learning, with Applications
摘要:
在预算学习问题中,我们可以在一组备选方案上进行实验(给定固定的实验预算),目标是选择一个具有最大可能预期收益的备选方案。这个问题的近似算法是由Guha和Munagala通过将各种备选方案结合在一起的线性规划四舍五入而开发的。在本文中,我们提出了这个问题的指数,我们称之为比率指数,这也保证了常数因子近似。基于索引的策略的优点是,可以为每个备选方案计算单个数字(即索引),而不管所有其他备选方案,并且对具有最高索引的备选方案进行实验。这类似于著名的Gittins指数的折扣多臂强盗问题。比率指数有几个有趣的结构特性。首先,我们证明了它可以在强多项式时间内计算。其次,我们证明了,适当的折扣因子,Gittins指数和我们的比率指数是常数因子近似对方,因此Gittins指数也给出了一个常数因子近似的预算学习问题。最后,我们证明了比率指数可以用来创建一个基于指数的策略,实现了O(1)-近似的有限时域版本的多臂强盗问题。此外,该政策不需要任何知识的地平线(而我们比较其性能与最优策略,是知道地平线)。这产生了以下令人惊讶的结果:有一个基于索引的策略,实现了多臂强盗问题的O(1)近似,忽略了潜在的折扣因子。
英文摘要:
In the budgeted learning problem, we are allowed to experiment on a set of alternatives (given a fixed experimentation budget) with the goal of picking a single alternative with the largest possible expected payoff. Approximation algorithms for this problem were developed by Guha and Munagala by rounding a linear program that couples the various alternatives together. In this paper we present an index for this problem, which we call the ratio index, which also guarantees a constant factor approximation. Index-based policies have the advantage that a single number (i.e. the index) can be computed for each alternative irrespective of all other alternatives, and the alternative with the highest index is experimented upon. This is analogous to the famous Gittins index for the discounted multi-armed bandit problem. The ratio index has several interesting structural properties. First, we show that it can be computed in strongly polynomial time. Second, we show that with the appropriate discount factor, the Gittins index and our ratio index are constant factor approximations of each other, and hence the Gittins index also gives a constant factor approximation to the budgeted learning problem. Finally, we show that the ratio index can be used to create an index-based policy that achieves an O(1)-approximation for the finite horizon version of the multi-armed bandit problem. Moreover, the policy does not require any knowledge of the horizon (whereas we compare its performance against an optimal strategy that is aware of the horizon). This yields the following surprising result: there is an index-based policy that achieves an O(1)-approximation for the multi-armed bandit problem, oblivious to the underlying discount factor.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。