arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2107.05289cs.LGcs.AIstat.ML
收藏

带采样成本的连续时间赌博机

Continuous Time Bandits With Sampling Costs

Rahul Vaze, Manjesh K. Hanawal

AI总结:

本文研究带采样成本的连续时间多臂赌博机问题,建立单臂和多臂情形下的遗憾下界,并提出达到下界(至多相差对数或常数因子)的学习算法。

摘要:

我们考虑一个连续时间多臂赌博机问题(CTMAB),其中学习者在给定区间内可以对臂进行任意次数的采样,并从每次采样中获得随机奖励,然而提高采样频率会带来加性惩罚/成本。因此,在获得较大奖励与因采样频率而产生的采样成本之间存在权衡。目标是设计一种最小化遗憾的学习算法,遗憾定义为预言机策略的收益与学习算法收益之差。CTMAB与通常的多臂赌博机问题(MAB)有根本不同,例如,在CTMAB中即使是单臂情形也不是平凡的,因为最优采样频率依赖于臂的均值,而该均值需要被估计。我们首先建立任何算法可达到的遗憾下界,然后提出达到该下界(至多相差对数因子)的算法。对于单臂情形,我们证明遗憾下界为$Ω((\log T)^2/μ)$,其中$μ$是臂的均值,$T$是时间范围。对于多臂情形,我们证明遗憾下界为$Ω((\log T)^2 μ/Δ^2)$,其中$μ$现在表示最优臂的均值,$Δ$是最优臂与次优臂均值之差。然后我们提出一个达到该界(至多相差常数项)的算法。

英文摘要:

We consider a continuous-time multi-arm bandit problem (CTMAB), where the learner can sample arms any number of times in a given interval and obtain a random reward from each sample, however, increasing the frequency of sampling incurs an additive penalty/cost. Thus, there is a tradeoff between obtaining large reward and incurring sampling cost as a function of the sampling frequency. The goal is to design a learning algorithm that minimizes regret, that is defined as the difference of the payoff of the oracle policy and that of the learning algorithm. CTMAB is fundamentally different than the usual multi-arm bandit problem (MAB), e.g., even the single-arm case is non-trivial in CTMAB, since the optimal sampling frequency depends on the mean of the arm, which needs to be estimated. We first establish lower bounds on the regret achievable with any algorithm and then propose algorithms that achieve the lower bound up to logarithmic factors. For the single-arm case, we show that the lower bound on the regret is $Ω((\log T)^2/μ)$, where $μ$ is the mean of the arm, and $T$ is the time horizon. For the multiple arms case, we show that the lower bound on the regret is $Ω((\log T)^2 μ/Δ^2)$, where $μ$ now represents the mean of the best arm, and $Δ$ is the difference of the mean of the best and the second-best arm. We then propose an algorithm that achieves the bound up to constant terms.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑