arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2106.01165math.OC
收藏

有限时域赌博机问题的分解-协调方法

Decomposition-Coordination Method for Finite Horizon Bandit Problems

Michel de Lara, Benjamin Heymann, Jean-Philippe Chancelier

AI总结:

针对多臂赌博机动态规划的指数复杂度,本文提出 DeCo 分解-协调启发式方法,将问题并行化为单臂问题,实现近线性计算时间,并在实验中优于经典算法且接近遗憾下界。

摘要:

最优求解多臂赌博机问题会遭遇维数灾难。事实上,随着臂数和时域增加,采用动态规划会导致计算时间指数增长。我们引入一种分解-协调启发式方法 DeCo,将初始问题转化为并行协调的单臂赌博机问题。因此,我们获得了本质上随臂数线性增长的计算时间。此外,该分解给出了遗憾的理论下界。对于双臂赌博机情形,动态规划给出精确解,而 DeCo 启发式方法几乎与之持平。此外,在至多100轮、20个臂的数值仿真中,DeCo 优于经典算法(Thompson sampling 和 Kullback-Leibler upper-confidence bound),并且在20个臂时几乎达到遗憾的理论下界。

英文摘要:

Optimally solving a multi-armed bandit problem suffers the curse of dimensionality. Indeed, resorting to dynamic programming leads to an exponential growth of computing time, as the number of arms and the horizon increase. We introduce a decompositioncoordination heuristic, DeCo, that turns the initial problem into parallelly coordinated one-armed bandit problems. As a consequence, we obtain a computing time which is essentially linear in the number of arms. In addition, the decomposition provides a theoretical lower bound on the regret. For the two-armed bandit case, dynamic programming provides the exact solution, which is almost matched by the DeCo heuristic. Moreover, in numerical simulations with up to 100 rounds and 20 arms, DeCo outperforms classic algorithms (Thompson sampling and Kullback-Leibler upper-confidence bound) and almost matches the theoretical lower bound on the regret for 20 arms.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑