关于遍历双臂强盗
On ergodic two-armed bandits
摘要：
设备有两个具有未知确定性回报的手臂，目标是渐进地识别最好的一个，而不会在另一个上花费太多时间。Narendra算法为此提供了一个随机过程。我们表明，在弱遍历假设下，这些确定性的回报，该程序最终选择最好的手臂（即，具有最大Cesaro极限），对于算法的适当步骤序列，概率为1。在i.i.d.的情况下。收益，这意味着一个“淬火”版本的“退火”结果的Lamberton，Pagès和Tarrès [安。更确切地说，如果$(η_{\ell,i})_{i\in \mathbb {N}}\in\{0,1\}^{\mathbb {N}}$、$\ell\in\{A,B\}$是如果我们在时间$i$进行博弈则我们将得到的确定性奖励序列，则我们在与Lamberton、Pagès和Tarrès [Ann. Appl. Probab. 14（2004）1424--1454]，代替i.i.d.通过假设经验平均值$\sum_{i=1}^nη_{A,i}/n$和$\sum_{i=1}^nη_{B,i}/n$随着$n$趋于无穷大而分别收敛到$θ_A$和$θ_B$，对于某些$\varepsilon >0$，速率至少为$1/(\log n)^{1+\varepsilon}$。我们还展示了一个易错性结果，即以正概率收敛到错误臂的选择，这意味着Lamberton，Pagès和Tarrès [Ann. Appl. Probab. 14（2004）1424--1454]在i.i.d.案子
英文摘要：
A device has two arms with unknown deterministic payoffs and the aim is to asymptotically identify the best one without spending too much time on the other. The Narendra algorithm offers a stochastic procedure to this end. We show under weak ergodic assumptions on these deterministic payoffs that the procedure eventually chooses the best arm (i.e., with greatest Cesaro limit) with probability one for appropriate step sequences of the algorithm. In the case of i.i.d. payoffs, this implies a "quenched" version of the "annealed" result of Lamberton, Pagès and Tarrès [Ann. Appl. Probab. 14 (2004) 1424--1454] by the law of iterated logarithm, thus generalizing it. More precisely, if $(η_{\ell,i})_{i\in \mathbb {N}}\in\{0,1\}^{\mathbb {N}}$, $\ell\in\{A,B\}$, are the deterministic reward sequences we would get if we played at time $i$, we obtain infallibility with the same assumption on nonincreasing step sequences on the payoffs as in Lamberton, Pagès and Tarrès [Ann. Appl. Probab. 14 (2004) 1424--1454], replacing the i.i.d. assumption by the hypothesis that the empirical averages $\sum_{i=1}^nη_{A,i}/n$ and $\sum_{i=1}^nη_{B,i}/n$ converge, as $n$ tends to infinity, respectively, to $θ_A$ and $θ_B$, with rate at least $1/(\log n)^{1+\varepsilon}$, for some $\varepsilon >0$. We also show a fallibility result, that is, convergence with positive probability to the choice of the wrong arm, which implies the corresponding result of Lamberton, Pagès and Tarrès [Ann. Appl. Probab. 14 (2004) 1424--1454] in the i.i.d. case.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
