平均场博弈学习的一般框架
A General Framework for Learning Mean-Field Games
摘要：
本文提出了一个通用的平均场博弈（GMFG）框架，用于大种群随机博弈中的同时学习和决策。它首先建立了一个唯一的纳什均衡存在这个GMFG，并证明了天真地结合强化学习与经典MFG中的不动点方法产生不稳定的算法。然后提出了基于值和基于策略的强化学习算法（分别为GMF-V和GMF-P），并分析了它们的收敛性和计算复杂性。均衡产品定价问题的实验表明，GMF-V-Q和GMF-P-TRPO，GMF-V和GMF-P的两个具体的实例，分别与Q-学习和TRPO，都是有效的和强大的GMFG设置。此外，他们的表现是优越的收敛速度，准确性和稳定性相比，现有的多智能体强化学习算法在$N$播放器设置。
英文摘要：
This paper presents a general mean-field game (GMFG) framework for simultaneous learning and decision-making in stochastic games with a large population. It first establishes the existence of a unique Nash Equilibrium to this GMFG, and demonstrates that naively combining reinforcement learning with the fixed-point approach in classical MFGs yields unstable algorithms. It then proposes value-based and policy-based reinforcement learning algorithms (GMF-V and GMF-P, respectively) with smoothed policies, with analysis of their convergence properties and computational complexities. Experiments on an equilibrium product pricing problem demonstrate that GMF-V-Q and GMF-P-TRPO, two specific instantiations of GMF-V and GMF-P, respectively, with Q-learning and TRPO, are both efficient and robust in the GMFG setting. Moreover, their performance is superior in convergence speed, accuracy, and stability when compared with existing algorithms for multi-agent reinforcement learning in the $N$-player setting.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
