基于深度强化学习的多主体微观一般均衡模型分析
Analyzing Micro-Founded General Equilibrium Models with Many Agents using Deep Reinforcement Learning
摘要：
实体经济可以被建模为一个有许多异质主体（如消费者、企业和政府）的序贯信息博弈。动态一般均衡（DGE）模型经常用于这种背景下的宏观经济分析。然而，使用现有的理论或计算方法，寻找一般均衡是具有挑战性的，特别是当使用微基础来建模个体代理时。在这里，我们展示了如何使用深度多智能体强化学习（MARL）来找到$ε$ -微基DGE模型中智能体类型的元均衡。虽然标准的MARL无法学习非平凡的解决方案，但我们的结构化学习课程能够稳定地收敛到有意义的解决方案。从概念上讲，我们的方法更灵活，不需要不切实际的假设，例如，连续的市场清算，通常用于分析易处理性。此外，我们的端到端GPU实现能够与大量RL经济代理进行快速实时收敛。我们在开放和封闭的真实商业周期（RBC）模型中展示了我们的方法，其中包括100名工人-消费者，10家公司和一名税收和再分配的社会规划师。我们通过最佳响应分析验证了学习的解决方案是$ε$ -元均衡，表明它们与经济直觉一致，并表明我们的方法可以学习开放RBC模型中定性不同的$ε$ -元均衡。因此，我们表明，硬件加速的MARL是一个很有前途的框架，用于模拟基于微观基础的经济复杂性。
英文摘要：
Real economies can be modeled as a sequential imperfect-information game with many heterogeneous agents, such as consumers, firms, and governments. Dynamic general equilibrium (DGE) models are often used for macroeconomic analysis in this setting. However, finding general equilibria is challenging using existing theoretical or computational methods, especially when using microfoundations to model individual agents. Here, we show how to use deep multi-agent reinforcement learning (MARL) to find $ε$-meta-equilibria over agent types in microfounded DGE models. Whereas standard MARL fails to learn non-trivial solutions, our structured learning curricula enable stable convergence to meaningful solutions. Conceptually, our approach is more flexible and does not need unrealistic assumptions, e.g., continuous market clearing, that are commonly used for analytical tractability. Furthermore, our end-to-end GPU implementation enables fast real-time convergence with a large number of RL economic agents. We showcase our approach in open and closed real-business-cycle (RBC) models with 100 worker-consumers, 10 firms, and a social planner who taxes and redistributes. We validate the learned solutions are $ε$-meta-equilibria through best-response analyses, show that they align with economic intuitions, and show our approach can learn a spectrum of qualitatively distinct $ε$-meta-equilibria in open RBC models. As such, we show that hardware-accelerated MARL is a promising framework for modeling the complexity of economies based on microfoundations.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
