arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2209.07330cs.LGecon.EMmath.STstat.MEstat.MLstat.TH
收藏

小间隙条件下基于上下文信息的最佳手臂识别

Best Arm Identification with Contextual Information under a Small Gap

the University of Tokyo(东京大学);CyberAgent, Inc.(赛博艾坚特公司);Tohoku University(东北大学);Brown University(布朗大学);University College London(伦敦大学学院)

Masahiro Kato, Masaaki Imaizumi, Takuya Ishihara, Toru Kitagawa

摘要:

我们研究了最佳臂识别(BAI)问题与固定的预算和上下文(协变量)的信息。在每一轮适应性实验中,在观察上下文信息后,我们使用过去的观察结果和当前上下文选择一个治疗组。我们的目标是确定最佳治疗组,这是一个治疗组的最大预期奖励边缘化的上下文分布,以最小的错误识别的概率。在这项研究中,我们考虑了一类非参数的强盗模型,收敛到位置移动模型时,间隙为零。首先,我们推导出小差距制度下的某类策略和强盗模型(潜在结果的概率模型)的错误识别概率的下界。小差距方案是最佳和次优治疗组之间的预期回报差距为零的情况,这对应于确定最佳治疗组的最坏情况之一。然后,我们开发了“随机抽样(RS)-增强逆概率加权(AIPW)”策略,这是渐近最优的,在这个意义上,在小缺口制度下,当预算趋于无穷大时,该策略下的错误识别概率与下限相匹配。RS-AIPW策略由跟踪目标样本分配比率的RS规则和使用AIPW估计器的推荐规则组成。

英文摘要:

We study the best-arm identification (BAI) problem with a fixed budget and contextual (covariate) information. In each round of an adaptive experiment, after observing contextual information, we choose a treatment arm using past observations and current context. Our goal is to identify the best treatment arm, which is a treatment arm with the maximal expected reward marginalized over the contextual distribution, with a minimal probability of misidentification. In this study, we consider a class of nonparametric bandit models that converge to location-shift models when the gaps go to zero. First, we derive lower bounds of the misidentification probability for a certain class of strategies and bandit models (probabilistic models of potential outcomes) under a small-gap regime. A small-gap regime is a situation where gaps of the expected rewards between the best and suboptimal treatment arms go to zero, which corresponds to one of the worst cases in identifying the best treatment arm. We then develop the ``Random Sampling (RS)-Augmented Inverse Probability weighting (AIPW) strategy,'' which is asymptotically optimal in the sense that the probability of misidentification under the strategy matches the lower bound when the budget goes to infinity in the small-gap regime. The RS-AIPW strategy consists of the RS rule tracking a target sample allocation ratio and the recommendation rule using the AIPW estimator.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑