Narendra-Shapiro强盗算法的遗憾界
Regret bounds for Narendra-Shapiro bandit algorithms
摘要：
Narendra-Shapiro（NS）算法是六十年代引入的带型算法（着眼于心理学或学习自动机中的应用），其收敛性在随机算法文献中得到了深入研究。在本文中，我们解决了以下问题：从\textit{regret}的角度来看，Narendra-Shapiro（NS）强盗算法的竞争力？在我们的主要结果中，我们表明，一些竞争的界限，可以得到这样的算法在其惩罚的版本（在\cite{Lamberton_Pages}介绍）。更准确地说，直到过度惩罚修改，与惩罚双臂强盗算法相关的伪遗憾$\bar{R}_n$一致地有界于$C \sqrt{n}$（其中$C$在本文中被明确表示）。\noindent我们还推广了现有的收敛性和收敛速度的结果，多臂的情况下的过度惩罚的强盗算法，包括收敛到不变测度的分段确定性马尔可夫过程（PSTH）经过适当的重整化。最后，在多臂情形下，给出了该系统的遍历性。
英文摘要：
Narendra-Shapiro (NS) algorithms are bandit-type algorithms that have been introduced in the sixties (with a view to applications in Psychology or learning automata), whose convergence has been intensively studied in the stochastic algorithm literature. In this paper, we adress the following question: are the Narendra-Shapiro (NS) bandit algorithms competitive from a \textit{regret} point of view? In our main result, we show that some competitive bounds can be obtained for such algorithms in their penalized version (introduced in \cite{Lamberton_Pages}). More precisely, up to an over-penalization modification, the pseudo-regret $\bar{R}_n$ related to the penalized two-armed bandit algorithm is uniformly bounded by $C \sqrt{n}$ (where $C$ is made explicit in the paper). \noindent We also generalize existing convergence and rates of convergence results to the multi-armed case of the over-penalized bandit algorithm, including the convergence toward the invariant measure of a Piecewise Deterministic Markov Process (PDMP) after a suitable renormalization. Finally, ergodic properties of this PDMP are given in the multi-armed case.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
