面向线性收益的双重稳健汤普森采样
Doubly robust Thompson sampling for linear payoffs
AI总结：
针对上下文赌博机中奖励缺失与遗憾分析复杂的问题，提出结合双重稳健估计器的DR汤普森采样算法，推导了不含上下文维度的更优遗憾界，实验验证其性能优于LinTS。
摘要：
赌博机问题的一大难点在于，仅能观测到被选中臂的随机奖励，其余臂的奖励始终缺失。臂的选择依赖于过往的上下文与奖励对，这进一步提升了遗憾值分析的复杂度。我们提出一种全新的多臂上下文赌博机算法，名为双重稳健（DR）汤普森采样，该算法将缺失数据领域的双重稳健估计器应用于带上下文的汤普森采样（\texttt{LinTS}，线性汤普森采样）。与此前采用缺失数据技术的研究（如\textit{Dimakopoulou等人2019年的平衡方法}、\textit{Kim等人2019年的双重方法}）不同，本算法支持一种全新的加性遗憾分解，可推导出更优的遗憾界，阶数为$\tilde{O}(ϕ^{-2}\sqrt{T})$，其中$ϕ^2$为上下文协方差矩阵的最小特征值。这是首个不包含上下文维度$d$、仅用$ϕ^2$表示的\texttt{LinTS}遗憾界。结合$ϕ^2$与$d$的关系，在多数实际场景中，该算法的遗憾界为$\tilde{O}(d\sqrt{T})$，较\texttt{LinTS}的遗憾界提升了$\sqrt{d}$倍。该方法的另一优势在于可利用所有上下文数据——无论对应臂是否被选中，从而规避了\texttt{LinTS}理论分析中“不饱和臂”的技术性定义。实验结果表明，该算法性能优于\texttt{LinTS}。
英文摘要：
A challenging aspect of the bandit problem is that a stochastic reward is observed only for the chosen arm and the rewards of other arms remain missing. The dependence of the arm choice on the past context and reward pairs compounds the complexity of regret analysis. We propose a novel multi-armed contextual bandit algorithm called Doubly Robust (DR) Thompson Sampling employing the doubly-robust estimator used in missing data literature to Thompson Sampling with contexts (\texttt{LinTS}). Different from previous works relying on missing data techniques (\citet{dimakopoulou2019balanced}, \citet{kim2019doubly}), the proposed algorithm is designed to allow a novel additive regret decomposition leading to an improved regret bound with the order of $\tilde{O}(ϕ^{-2}\sqrt{T})$, where $ϕ^2$ is the minimum eigenvalue of the covariance matrix of contexts. This is the first regret bound of \texttt{LinTS} using $ϕ^2$ without the dimension of the context, $d$. Applying the relationship between $ϕ^2$ and $d$, the regret bound of the proposed algorithm is $\tilde{O}(d\sqrt{T})$ in many practical scenarios, improving the bound of \texttt{LinTS} by a factor of $\sqrt{d}$. A benefit of the proposed method is that it utilizes all the context data, chosen or not chosen, thus allowing to circumvent the technical definition of unsaturated arms used in theoretical analysis of \texttt{LinTS}. Empirical studies show the advantage of the proposed algorithm over \texttt{LinTS}.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
