arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 1901.07924cs.LGstat.ML
收藏

具有不同用户偏好的在线学习

Online Learning with Diverse User Preferences

Chao Gan, Jing Yang, Ruida Zhou, Cong Shen

摘要:

在本文中,我们研究了不同的用户偏好对学习的随机多臂强盗(MAB)框架下的影响。我们的目标是表明,当用户的喜好是足够多样的,每个手臂可以为某些用户的最佳,O(log T)的遗憾,通过探索次优武器下的标准随机MAB设置可以减少到一个常数。我们的直觉是,为了实现次线性后悔,最佳手臂被拉动的次数应该在时间上线性缩放;当所有手臂对于某些用户都是最佳的并且经常被拉动时,估计的手臂统计数据可以快速收敛到它们的真实值,从而大大减少探索的需要。我们把问题转化为一个随机线性土匪模型,其中用户的偏好和武器的状态被建模为{独立和相同的分布(i.i.d)} d维随机向量。在每个时隙开始时接收到用户偏好向量之后,学习者拉动手臂并接收作为偏好向量和手臂状态向量的线性乘积的奖励。我们还假设,一旦被拉动,被拉动的手臂的状态就会向学习者显示。我们提出了一个加权置信上限(W-UCB)算法,并表明,它可以实现一个恒定的遗憾时,用户的喜好是足够多样的。W-UCB在一般设置下的性能也完全表征和合成数据验证。

英文摘要:

In this paper, we investigate the impact of diverse user preference on learning under the stochastic multi-armed bandit (MAB) framework. We aim to show that when the user preferences are sufficiently diverse and each arm can be optimal for certain users, the O(log T) regret incurred by exploring the sub-optimal arms under the standard stochastic MAB setting can be reduced to a constant. Our intuition is that to achieve sub-linear regret, the number of times an optimal arm being pulled should scale linearly in time; when all arms are optimal for certain users and pulled frequently, the estimated arm statistics can quickly converge to their true values, thus reducing the need of exploration dramatically. We cast the problem into a stochastic linear bandits model, where both the users preferences and the state of arms are modeled as {independent and identical distributed (i.i.d)} d-dimensional random vectors. After receiving the user preference vector at the beginning of each time slot, the learner pulls an arm and receives a reward as the linear product of the preference vector and the arm state vector. We also assume that the state of the pulled arm is revealed to the learner once its pulled. We propose a Weighted Upper Confidence Bound (W-UCB) algorithm and show that it can achieve a constant regret when the user preferences are sufficiently diverse. The performance of W-UCB under general setups is also completely characterized and validated with synthetic data.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑