arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2202.11735stat.MLcs.LGmath.STstat.TH
收藏

用于随机线性赌博机的截断 LinUCB

Truncated LinUCB for Stochastic Linear Bandits

Department of Mathematics and Statistics, Queen’s University(皇后大学数学与统计学系)

Yanglei Song, Meng zhou

AI总结:

本文提出截断 LinUCB 算法(Tr-LinUCB),通过在截断时间后停止探索并转为纯利用,解决了 LinUCB 因过度探索导致的遗憾次优问题,并证明其在低维情形下达到 O(d log(T)) 的最优遗憾速率。

摘要:

本文研究具有有限臂数的上下文赌博机问题,其中上下文是独立同分布的 d 维随机向量,期望奖励对臂参数和上下文都是线性的。LinUCB 算法在相关线性赌博机问题中接近极小极大最优,但由于其过度探索,其累积遗憾在维度 d 和时间范围 T 上都是次优的。本文提出 LinUCB 的截断版本,称为“Tr-LinUCB”,该算法在截断时间 S 之前遵循 LinUCB,之后执行纯利用。研究表明,如果 S = Cd log(T),其中 C 是足够大的常数,则 Tr-LinUCB 算法可实现 O(d log(T)) 的遗憾,并建立了匹配的下界,证明了在低维情形下 Tr-LinUCB 在 d 和 T 上的速率最优性。此外,如果 S = d log^κ(T)(κ > 1),则与最优值的损失是一个乘性的 log log(T) 因子,该因子不依赖于 d。Tr-LinUCB 在选择截断时间时对超调不敏感,这一特性具有重要的实际意义。

英文摘要:

This paper considers contextual bandits with a finite number of arms, where the contexts are independent and identically distributed $d$-dimensional random vectors, and the expected rewards are linear in both the arm parameters and contexts. The LinUCB algorithm, which is near minimax optimal for related linear bandits, is shown to have a cumulative regret that is suboptimal in both the dimension $d$ and time horizon $T$, due to its over-exploration. A truncated version of LinUCB is proposed and termed "Tr-LinUCB", which follows LinUCB up to a truncation time $S$ and performs pure exploitation afterwards. The Tr-LinUCB algorithm is shown to achieve $O(d\log(T))$ regret if $S = Cd\log(T)$ for a sufficiently large constant $C$, and a matching lower bound is established, which shows the rate optimality of Tr-LinUCB in both $d$ and $T$ under a low dimensional regime. Further, if $S = d\log^κ(T)$ for some $κ>1$, the loss compared to the optimal is a multiplicative $\log\log(T)$ factor, which does not depend on $d$. This insensitivity to overshooting in choosing the truncation time of Tr-LinUCB is of practical importance.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑