arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 1612.01367cs.LG
收藏

一种基于分层结构的渐近最优上下文Bandit算法

An Asymptotically Optimal Contextual Bandit Algorithm Using Hierarchical Structures

Bilkent University(比尔肯特大学);Massachusetts Institute of Technology(麻省理工学院);Sabancı University(萨班哲大学)

Mohammadreza Mohaghegh Neyshabouri, Kaan Gokcesu, Huseyin Ozkan, Suleyman S. Kozat

摘要:

我们提出了在线算法顺序学习的上下文多臂强盗设置。我们的方法是划分上下文空间,然后以数据驱动的方式将所有可能的分区区域和强盗臂集之间的映射进行优化组合。我们表明,在我们的方法中,最好的映射是能够近似的最佳手臂选择政策,以任何所需的程度下温和的Lipschitz条件。因此,我们设计我们的算法的基础上的最佳自适应组合和渐近实现的最佳映射以及最佳的手臂选择政策的性能。即使在对抗性环境中,这种最优性也是可以保证的,因为我们不依赖于任何关于上下文或强盗武器损失的统计假设。此外,我们设计了有效的实现我们的算法在各种层次划分结构,如字典或任意位置分裂和二叉树(和其他几个分区的例子)。例如,在二叉树分区的情况下,计算复杂性仅与最精细分区中的区域数量成对数线性关系。总之,我们提供了显着的性能改进,通过引入上界(w.r.t.最佳臂选择策略),这些策略在数学上被证明在每轮平均损失意义上以比现有技术更快的速率消失。我们的实验工作广泛地覆盖了从强盗设置到真实和合成数据的多类分类的各种场景。在这些实验中,我们表明,我们的算法是非常优越的国家的最先进的技术,同时保持引入的数学保证和计算体面的可扩展性。

英文摘要:

We propose online algorithms for sequential learning in the contextual multi-armed bandit setting. Our approach is to partition the context space and then optimally combine all of the possible mappings between the partition regions and the set of bandit arms in a data driven manner. We show that in our approach, the best mapping is able to approximate the best arm selection policy to any desired degree under mild Lipschitz conditions. Therefore, we design our algorithms based on the optimal adaptive combination and asymptotically achieve the performance of the best mapping as well as the best arm selection policy. This optimality is also guaranteed to hold even in adversarial environments since we do not rely on any statistical assumptions regarding the contexts or the loss of the bandit arms. Moreover, we design efficient implementations for our algorithms in various hierarchical partitioning structures such as lexicographical or arbitrary position splitting and binary trees (and several other partitioning examples). For instance, in the case of binary tree partitioning, the computational complexity is only log-linear in the number of regions in the finest partition. In conclusion, we provide significant performance improvements by introducing upper bounds (w.r.t. the best arm selection policy) that are mathematically proven to vanish in the average loss per round sense at a faster rate compared to the state-of-the-art. Our experimental work extensively covers various scenarios ranging from bandit settings to multi-class classification with real and synthetic data. In these experiments, we show that our algorithms are highly superior over the state-of-the-art techniques while maintaining the introduced mathematical guarantees and a computationally decent scalability.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑