使用有限记忆协作学习最佳选项
Collaboratively Learning the Best Option, Using Bounded Memory
摘要：
我们认为多臂强盗问题的社会群体中，每个人都有界的记忆和共享的共同目标，学习最好的武器/选项。如果最终（如$t \to \infty$）只拉平均奖励最高的那只手臂，我们就说它学会了最佳选择。虽然这一目标对于一个孤立的个体来说是不可能实现的，但我们表明，在社会群体中，这一目标可以在社会说服的帮助下轻松实现，即，通信具体来说，我们研究的学习动力学，其中一个人顺序决定在哪只手臂拉下不仅根据其私人奖励反馈，但也随机选择的同行提供的建议。我们的学习动力学很难通过显式概率计算进行分析，这是由于社会互动引起的随机依赖性。相反，我们采用统计物理学中的平均场近似方法，并表明：（1）以概率$\to 1$作为社会群体大小$N \to \infty $，社会群体中的每个人都会学习最佳选择。(2)在任意有限时间范围$[0, T]$上，随着$t$的增加，偏好最佳选项的个体比例以高概率（在$N$中）以指数速度增长到1（$t\in [0, T]$）。我们的平均场分析的一个主要创新是一个简单而强大的技术来处理吸收状态的交换限制$N \to \infty$和$t \to \infty $。平均场近似方法允许我们通过一个确定性和平滑的轨迹来近似我们学习动态的概率样本路径，该轨迹对应于一个行为良好的常微分方程（ODE）系统的唯一解。这样的近似是期望的，因为常微分方程系统的分析比原始随机系统的分析相对容易。
英文摘要：
We consider multi-armed bandit problems in social groups wherein each individual has bounded memory and shares the common goal of learning the best arm/option. We say an individual learns the best option if eventually (as $t \to \infty$) it pulls only the arm with the highest average reward. While this goal is provably impossible for an isolated individual, we show that, in social groups, this goal can be achieved easily with the aid of social persuasion, i.e., communication. Specifically, we study the learning dynamics wherein an individual sequentially decides on which arm to pull next based on not only its private reward feedback but also the suggestions provided by randomly chosen peers. Our learning dynamics are hard to analyze via explicit probabilistic calculations due to the stochastic dependency induced by social interaction. Instead, we employ the mean-field approximation method from statistical physics and we show: (1) With probability $\to 1$ as the social group size $N \to \infty $, every individual in the social group learns the best option. (2) Over an arbitrary finite time horizon $[0, T]$, with high probability (in $N$), the fraction of individuals that prefer the best option grows to 1 exponentially fast as $t$ increases ($t\in [0, T]$). A major innovation of our mean-filed analysis is a simple yet powerful technique to deal with absorbing states in the interchange of limits $N \to \infty$ and $t \to \infty $. The mean-field approximation method allows us to approximate the probabilistic sample paths of our learning dynamics by a deterministic and smooth trajectory that corresponds to the unique solution of a well-behaved system of ordinary differential equations (ODEs). Such an approximation is desired because the analysis of a system of ODEs is relatively easier than that of the original stochastic system.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
