arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2302.13252cs.LG
收藏

超越可实现性的无遗憾线性老虎机

No-Regret Linear Bandits beyond Realizability

University of California Santa Barbara(加州大学圣巴巴拉分校)

Chong Liu, Ming Yin, Yu-Xiang Wang

AI总结:

针对线性老虎机的奖励函数误设问题,提出与次优性差距挂钩的自然误设模型,证明经典LinUCB算法对该类误设天然鲁棒,可取得接近最优的√T阶遗憾,突破了此前近乎线性的遗憾下界。

摘要:

我们研究底层奖励函数不满足线性假设时的线性老虎机问题。现有研究依赖统一的误设参数$\varepsilon$,用于衡量最佳线性近似的上确界范数误差,这导致当$\varepsilon>0$时,不可避免地会产生线性遗憾。我们提出一种更自然的误设模型,仅要求每个输入$x$处的近似误差与$x$处的次优性差距成正比。该模型体现了如下直觉:对于优化问题,接近最优的区域更为重要,而在次优区域我们可以容忍更大的近似误差。令人惊讶的是,我们证明了为可实现场景设计的经典LinUCB算法,对这种差距调整型误设天然具备鲁棒性。对于此前已知最佳遗憾几乎与时域$T$呈线性关系的问题,该算法能取得接近最优的$\sqrt{T}$阶遗憾。从技术层面看,我们的证明依赖一种新颖的自界论证,将误设导致的遗憾部分用遗憾本身来界定。

英文摘要:

We study linear bandits when the underlying reward function is not linear. Existing work relies on a uniform misspecification parameter $ε$ that measures the sup-norm error of the best linear approximation. This results in an unavoidable linear regret whenever $ε> 0$. We describe a more natural model of misspecification which only requires the approximation error at each input $x$ to be proportional to the suboptimality gap at $x$. It captures the intuition that, for optimization problems, near-optimal regions should matter more and we can tolerate larger approximation errors in suboptimal regions. Quite surprisingly, we show that the classical LinUCB algorithm -- designed for the realizable case -- is automatically robust against such gap-adjusted misspecification. It achieves a near-optimal $\sqrt{T}$ regret for problems that the best-known regret is almost linear in time horizon $T$. Technically, our proof relies on a novel self-bounding argument that bounds the part of the regret due to misspecification by the regret itself.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑