arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2106.13755math.OCcs.LG
收藏

平均场博弈的强化学习及其在经济学中的应用

Reinforcement Learning for Mean Field Games, with Applications to Economics

University of California Santa Barbara(加州大学圣巴巴拉分校);Princeton University(普林斯顿大学)

Andrea Angiuli, Jean-Pierre Fouque, Mathieu Lauriere

摘要:

平均场博弈(MFG)和平均场控制问题(MFC)是研究具有连续个体的博弈中的纳什均衡或社会最优解的框架。这些问题可以用来近似竞争或合作对策与大量的有限数量的代理人,并已发现了广泛的应用,特别是在经济学。近年来,MFG和MFC中的学习问题引起了人们的兴趣,无论是作为一种计算解决方案的方法,还是作为一种模拟大量学习者如何收敛到均衡的方法。特别感兴趣的是智能体不知道模型的设置,这导致了强化学习(RL)方法的发展。在回顾了有关这一主题的文献后,我们提出了一种针对MFG和MFC的RL双时标方法,该方法依赖于统一的Q学习算法。该方法的主要新颖之处是同时更新动作值函数和分布,但具有不同的速率,以无模型的方式。根据两个学习率的比率,算法学习MFG或MFC解决方案。为了说明这种方法,我们将其应用到一个平均场问题的累积消费在有限的地平线与HARA效用函数,和一个交易者的最佳清算问题。

英文摘要:

Mean field games (MFG) and mean field control problems (MFC) are frameworks to study Nash equilibria or social optima in games with a continuum of agents. These problems can be used to approximate competitive or cooperative games with a large finite number of agents and have found a broad range of applications, in particular in economics. In recent years, the question of learning in MFG and MFC has garnered interest, both as a way to compute solutions and as a way to model how large populations of learners converge to an equilibrium. Of particular interest is the setting where the agents do not know the model, which leads to the development of reinforcement learning (RL) methods. After reviewing the literature on this topic, we present a two timescale approach with RL for MFG and MFC, which relies on a unified Q-learning algorithm. The main novelty of this method is to simultaneously update an action-value function and a distribution but with different rates, in a model-free fashion. Depending on the ratio of the two learning rates, the algorithm learns either the MFG or the MFC solution. To illustrate this method, we apply it to a mean field problem of accumulated consumption in finite horizon with HARA utility function, and to a trader's optimal liquidation problem.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑