arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2610.18258cs.LG
收藏

可上下文分块的多臂老虎机的最优在线悔值

Optimal Online Regret for Context-Lumpable Bandits

The Hong Kong University of Science and Technology(香港科技大学)

Chenyu Lu, Zihan Zhang

AI总结:

针对未知上下文分块结构的随机上下文多臂老虎机,提出一种算法,将期望悔值上界提升r倍,匹配Lee等人2023年的下界,通过跨上下文消除与多尺度探索实现近最优性能。

摘要:

我们研究具有未知上下文分块结构的随机上下文多臂老虎机问题。S个上下文被划分为r个块,每个臂在同一块内的所有上下文中具有相同的平均奖励。在任意上下文分布且无分离假设的情况下,我们设计了一种算法,其期望悔值为\\(\widetilde{O}(\sqrt{r(S+K)T})\\),与Lee等人(2023)的下界相比仅差对数因子,且将他们之前的上界提高了r倍。该算法在二进精度级别上运行,使用基线策略和基于置信度的早停来控制探索,随后执行跨上下文消除:由于同一块内的奖励相同,一个选中的臂会消除该块内所有剩余记录,每个级别最多需要r次选择。多尺度探索为频繁块分配更精细的精度,为稀有块分配更粗糙的精度,为每个块生成近最优臂,同时保持加权策略误差较小。

英文摘要:

We study stochastic contextual bandits with an unknown context-lumpable structure. The $S$ contexts are partitioned into $r$ blocks, and every arm has the same mean reward across contexts in the same block. Under arbitrary context distributions and without separation assumptions, we design an algorithm with expected regret $\widetilde{O}(\sqrt{r(S+K)T})$, matching the lower bound of Lee et al. (2023) up to logarithmic factors and improving their previous upper bound by a factor of $r$. The algorithm operates over dyadic accuracy levels, using a baseline policy and confidence-based early stopping to control exploration. It then performs cross-context elimination: since rewards are identical within a block, one selected arm eliminates all surviving records from that block, requiring at most $r$ selections at each level. Multi-scale exploration assigns finer accuracies to frequent blocks and coarser accuracies to rare blocks, yielding a near-optimal arm for every block while keeping the weighted policy error small.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑