通过强化学习与智能体建模模拟理性对经济的影响
Simulating the Economic Impact of Rationality through Reinforcement Learning and Agent-Based Modelling
AI总结:
本文提出理性宏观 ABM(R-MABM)框架,将多智能体强化学习引入经济仿真,发现理性智能体能自发学习多种利润最大化策略并分化群体,其数量增加可提升总产出但可能加剧不稳定性。
摘要:
基于智能体的模型(ABMs)是经济学中用于克服传统一般均衡假设框架某些局限性的仿真模型。然而,ABM 中的智能体遵循预先设定的“有限理性”行为规则,这些规则的设计可能繁琐且难以论证其合理性。本文利用多智能体强化学习(RL)来扩展 ABM 的能力,引入“完全理性”的智能体,这些智能体通过与环境交互并最大化奖励函数来学习其策略。具体而言,我们通过扩展经济学文献中的一个典型宏观 ABM,提出了一个“理性宏观 ABM”(R-MABM)框架。我们表明,逐步用经过利润最大化训练的 RL 智能体替换模型中的 ABM 企业,可以用于研究理性对经济的影响。我们发现,RL 智能体会自发地学习三种不同的利润最大化策略,最优策略取决于市场竞争程度和理性水平。我们还发现,具有独立策略且彼此之间无法通信的 RL 智能体,会自发地学习分化为不同的策略群体,从而提高市场势力和总体利润。最后,我们发现经济中理性(RL)智能体数量的增加总是会改善以总产出衡量的宏观经济环境。根据具体理性策略的不同,这可能以更高的不稳定性为代价。我们的 R-MABM 框架支持稳定的多智能体学习,并以开源形式提供,代表了扩展经济仿真器的一个有原则且稳健的方向。
英文摘要:
Agent-based models (ABMs) are simulation models used in economics to overcome some of the limitations of traditional frameworks based on general equilibrium assumptions. However, agents within an ABM follow predetermined 'bounded rational' behavioural rules which can be cumbersome to design and difficult to justify. Here we leverage multi-agent reinforcement learning (RL) to expand the capabilities of ABMs with the introduction of 'fully rational' agents that learn their policy by interacting with the environment and maximising a reward function. Specifically, we propose a 'Rational macro ABM' (R-MABM) framework by extending a paradigmatic macro ABM from the economic literature. We show that gradually substituting ABM firms in the model with RL agents, trained to maximise profits, allows for studying the impact of rationality on the economy. We find that RL agents spontaneously learn three distinct strategies for maximising profits, with the optimal strategy depending on the level of market competition and rationality. We also find that RL agents with independent policies, and without the ability to communicate with each other, spontaneously learn to segregate into different strategic groups, thus increasing market power and overall profits. Finally, we find that a higher number of rational (RL) agents in the economy always improves the macroeconomic environment as measured by total output. Depending on the specific rational policy, this can come at the cost of higher instability. Our R-MABM framework allows for stable multi-agent learning, is available in open source, and represents a principled and robust direction to extend economic simulators.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。