arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 1306.1149cs.DS
收藏

随机背包和多臂Bandit近似算法的改进和推广:完整版

Improvements and Generalizations of Stochastic Knapsack and Multi-Armed Bandit Approximation Algorithms: Full Version

Will Ma

摘要:

研究了具有报酬马氏链的多臂强盗问题。在有限时间内,著名的Gittins指数不适用,精确解是棘手的。我们提供了一个更一般的模型,其中包括马尔可夫决策过程和非单位转移时间的近似算法。当允许抢占时,我们提供了一个(1/2-eps)近似,以及一个例子表明这是紧的。当不允许抢占时,我们提供了1/12近似,当过渡时间为1时,它提高到4/27近似。我们的模型包括古普塔等人的马尔可夫Bandits模型,Dean,Goemans和Vondrak的随机背包模型,以及Guha和Munagala的学习模型,我们的算法改进了所有三个领域的现有结果。在我们的分析中,我们遇到并克服了一个新的障碍-一个算法,可证明存在通过多面体参数,但不能在多项式时间。

英文摘要:

We study the multi-armed bandit problem with arms which are Markov chains with rewards. In the finite-horizon setting, the celebrated Gittins indices do not apply, and the exact solution is intractable. We provide approximation algorithms for a more general model which includes Markov decision processes and non-unit transition times. When preemption is allowed, we provide a (1/2-eps)-approximation, along with an example showing this is tight. When preemption isn't allowed, we provide a 1/12-approximation, which improves to a 4/27-approximation when transition times are unity. Our model encompasses the Markovian Bandits model of Gupta et al, the Stochastic Knapsack model of Dean, Goemans, and Vondrak, and the Budgeted Learning model of Guha and Munagala, and our algorithms improve existing results in all three areas. In our analysis, we encounter and overcome to our knowledge a novel obstacle - an algorithm that provably exists via polyhedral arguments, but cannot be found in polynomial time.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑