arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2610.13407cs.LG
收藏

基于持久候选集加速线性多臂老虎机算法

Accelerating Linear Bandits with Persistent Candidate Sets

Mohamed bin Zayed University of Artificial Intelligence(穆罕默德·本·扎耶德人工智能大学);HSE University(高等经济大学);RIKEN AIP(理化学研究所人工智能项目)

Bingkui Tong, Daria Korovaitceva, Marina Sheshukova, Sergey Samsonov, Junpei Komiyama

AI总结:

本文提出持久候选LinUCB算法,通过减少线性多臂老虎机每轮评分评估次数实现加速,建立了相关遗憾界,实验验证其加速效果且可扩展至LinUCB之外。

摘要:

在具有大量固定K个动作的线性多臂老虎机问题中,穷举乐观评分会导致动作选择成本高昂,每轮需要进行O(K)次评分评估。我们提出了持久候选LinUCB(\textsc{PC-LinUCB}),该算法每轮维护k个活跃动作,并从剩余的K-k个动作中均匀采样q个挑战者。通过仅对这k+q个候选进行评分,\textsc{PC-LinUCB}将每轮的评分评估次数减少到O(k+q)。我们在固定置信度下建立了与问题无关和与问题相关的高概率遗憾界,该界由标准线性多臂老虎机学习项和阶为O((K-k)log(K-k)/q)的附加发现成本组成,值得注意的是,该附加成本不会随时间跨度增长。在固定间隙和足够长的时间跨度下,我们在均匀挑战者采样下建立了Ω((K-k)/q)的遗憾下界,表明部分候选访问本身会带来不可避免的遗憾成本。实验表明,\textsc{PC-LinUCB}在保持可比遗憾的同时大幅加速了LinUCB,且持久候选机制可扩展到LinUCB之外。

英文摘要:

Exhaustive optimistic scoring can make action selection costly in linear bandits with a large fixed set of $K$ actions, requiring $\mathcal{O}(K)$ score evaluations per round. We introduce Persistent Candidate LinUCB (\textsc{PC-LinUCB}), which maintains $k$ active actions and samples $q$ challengers uniformly from the remaining $K-k$ actions at each round. By scoring only these $k+q$ candidates, \textsc{PC-LinUCB} reduces the number of score evaluations per round to $\mathcal{O}(k+q)$. We establish problem-independent and problem-dependent high-probability regret bounds consisting of a standard linear-bandit learning term and an additive discovery cost of order $\mathcal{O}\left((K-k)\log(K-k)/q\right)$ at fixed confidence. Notably, this additional cost does not grow with the horizon. For fixed gaps and sufficiently long horizons, we establish an $Ω\left((K-k)/q\right)$ regret lower bound under uniform challenger sampling, showing that partial candidate access itself imposes an unavoidable regret cost. Experiments show that \textsc{PC-LinUCB} substantially accelerates LinUCB while maintaining comparable regret, and that the persistent candidate mechanism can extend beyond LinUCB.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑