arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 1901.09585math.OC
收藏

学习平均场博弈

Learning Mean-Field Games

Xin Guo, Anran Hu, Renyuan Xu, Junzi Zhang

摘要:

本文提出了一个通用的均值场博弈(GMFG)框架,用于大种群随机对策的同时学习和决策。它首先建立了一个唯一的纳什均衡存在这个GMFG,并解释了天真地结合Q学习与经典MFG中的不动点方法产生不稳定的算法。在此基础上,提出了一种基于玻尔兹曼策略的Q学习算法(GMF-Q),并分析了算法的收敛性和计算复杂性。对重复广告拍卖问题的实验结果表明,该算法在收敛性和学习精度方面具有较好的鲁棒性和有效性。此外,它的性能是优越的收敛性,稳定性和学习能力,与现有的多智能体强化学习算法相比。

英文摘要:

This paper presents a general mean-field game (GMFG) framework for simultaneous learning and decision-making in stochastic games with a large population. It first establishes the existence of a unique Nash Equilibrium to this GMFG, and explains that naively combining Q-learning with the fixed-point approach in classical MFGs yields unstable algorithms. It then proposes a Q-learning algorithm with Boltzmann policy (GMF-Q), with analysis of convergence property and computational complexity. The experiments on repeated Ad auction problems demonstrate that this GMF-Q algorithm is efficient and robust in terms of convergence and learning accuracy. Moreover, its performance is superior in convergence, stability, and learning ability, when compared with existing algorithms for multi-agent reinforcement learning.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑