arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 1803.05028cs.MA
收藏

多策略代理系统中的分散学习

Decentralised Learning in Systems with Many, Many Strategic Agents

David Mguni, Joel Jennings, Enrique Munoz de Cote

摘要:

虽然多智能体强化学习可以处理策略交互实体的系统,但它目前在可扩展性方面失败,并且缺乏严格的收敛保证。至关重要的是,多智能体系统中的学习可能会变得棘手,因为随着智能体数量的增加,状态-动作空间的大小会爆炸。在本文中,我们提出了一种方法来计算闭环最优策略的多智能体系统的规模独立的代理的数量。这使我们能够显示,第一次,成功的收敛到最佳行为的系统与无限数量的相互作用的自适应学习器。研究N-玩家随机博弈的渐近制度,我们设计了一个学习协议,保证收敛到均衡的政策,即使代理的数量是非常大的。我们的方法是无模型和完全分散的,使每个代理只需要观察其本地状态信息和实现的奖励。我们验证这些理论结果表明收敛到纳什均衡政策的应用,从经济学和控制理论与成千上万的战略互动代理。

英文摘要:

Although multi-agent reinforcement learning can tackle systems of strategically interacting entities, it currently fails in scalability and lacks rigorous convergence guarantees. Crucially, learning in multi-agent systems can become intractable due to the explosion in the size of the state-action space as the number of agents increases. In this paper, we propose a method for computing closed-loop optimal policies in multi-agent systems that scales independently of the number of agents. This allows us to show, for the first time, successful convergence to optimal behaviour in systems with an unbounded number of interacting adaptive learners. Studying the asymptotic regime of N-player stochastic games, we devise a learning protocol that is guaranteed to converge to equilibrium policies even when the number of agents is extremely large. Our method is model-free and completely decentralised so that each agent need only observe its local state information and its realised rewards. We validate these theoretical results by showing convergence to Nash-equilibrium policies in applications from economics and control theory with thousands of strategically interacting agents.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑