多智能体赌博机的闲聊插入-消除算法
The Gossiping Insert-Eliminate Algorithm for Multi-Agent Bandits
AI总结：
本文针对去中心化多智能体多臂赌博机问题，提出基于闲聊通信的插入-消除算法，证明少量通信即可使每个智能体的遗憾降低约 N 倍，并给出遗憾与通信的权衡界。
摘要：
我们考虑一个去中心化的多智能体多臂赌博机（Multi Armed Bandit, MAB）设置，该设置由 $N$ 个智能体组成，它们求解同一个 MAB 实例以最小化各自的累积遗憾。在我们的模型中，智能体通过在任意连通图上的成对闲聊（gossip）式通信交换消息来进行协作。我们提出了两种新算法，其中每个智能体只从所有臂的一个子集中进行选择。智能体利用通信媒介仅推荐臂的 ID（而不是样本），从而更新它们所选择臂的集合。我们证明，如果智能体通过任意连通的成对闲聊机制通信 $Ω(\log(T))$ 次，那么与没有协作的情况相比，每个智能体的遗憾在数量级上减小了 $N$ 倍。此外，我们表明通信约束只对我们的算法的遗憾产生二阶影响。然后，我们分析遗憾的这一二阶项，以推导遗憾与通信之间权衡的界。最后，我们通过实验评估了我们的算法，并得出结论：这些见解是根本性的，而不是我们推导出的界所带来的人为产物。我们还给出了一个下界，表明即使在没有任何通信约束的情况下，我们的算法所获得的遗憾缩放也无法进一步改进。因此，我们的结果表明，即使智能体之间只有最低限度的协作，也能极大地降低所有智能体的遗憾。
英文摘要：
We consider a decentralized multi-agent Multi Armed Bandit (MAB) setup consisting of $N$ agents, solving the same MAB instance to minimize individual cumulative regret. In our model, agents collaborate by exchanging messages through pairwise gossip style communications on an arbitrary connected graph. We develop two novel algorithms, where each agent only plays from a subset of all the arms. Agents use the communication medium to recommend only arm-IDs (not samples), and thus update the set of arms from which they play. We establish that, if agents communicate $Ω(\log(T))$ times through any connected pairwise gossip mechanism, then every agent's regret is a factor of order $N$ smaller compared to the case of no collaborations. Furthermore, we show that the communication constraints only have a second order effect on the regret of our algorithm. We then analyze this second order term of the regret to derive bounds on the regret-communication tradeoffs. Finally, we empirically evaluate our algorithm and conclude that the insights are fundamental and not artifacts of our bounds. We also show a lower bound which gives that the regret scaling obtained by our algorithm cannot be improved even in the absence of any communication constraints. Our results thus demonstrate that even a minimal level of collaboration among agents greatly reduces regret for all agents.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
