arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2104.12895cs.GTcs.AIcs.LGcs.MAecon.TH
收藏

深度强化学习求解纳什均衡的计算性能

Computational Performance of Deep Reinforcement Learning to find Nash Equilibria

Program on Energy and Sustainable Development (PESD), Stanford University(能源与可持续发展项目,斯坦福大学);Institute for Sustainable Economic Development, University of Natural Resources and Life Sciences(可持续经济发展的研究所,自然资源与生命科学大学)

Christoph Graf, Viktor Zobernig, Johannes Schmidt, Claude Klöckl

AI总结:

本文系统评估 DDPG 在价格竞争中学习 Bertrand 纳什均衡时的参数敏感性,发现合适配置可实现最高 99% 的收敛率。

摘要:

我们测试了深度确定性策略梯度(DDPG)这一能够处理连续状态与动作空间的深度强化学习算法,在企业进行价格竞争的情境中学习纳什均衡的性能。此类算法通常被视为无模型方法,因为它们不需要转移概率函数(例如马尔可夫博弈中的情形)或预先设定的函数形式。尽管属于无模型方法,算法的各个步骤仍使用大量参数,例如学习率、记忆缓冲区、状态空间维度设置、归一化或噪声衰减率;本研究的目的是系统测试这些参数配置对收敛至解析推导的 Bertrand 均衡的影响。我们发现,某些参数选择可使收敛率最高达到 99%。这种可靠的收敛性可能使该方法成为研究企业策略行为的有用工具,即使在更复杂的情境中也是如此。关键词:Bertrand 均衡、统一价格拍卖中的竞争、深度确定性策略梯度算法、参数敏感性分析。

英文摘要:

We test the performance of deep deterministic policy gradient (DDPG), a deep reinforcement learning algorithm, able to handle continuous state and action spaces, to learn Nash equilibria in a setting where firms compete in prices. These algorithms are typically considered model-free because they do not require transition probability functions (as in e.g., Markov games) or predefined functional forms. Despite being model-free, a large set of parameters are utilized in various steps of the algorithm. These are e.g., learning rates, memory buffers, state-space dimensioning, normalizations, or noise decay rates and the purpose of this work is to systematically test the effect of these parameter configurations on convergence to the analytically derived Bertrand equilibrium. We find parameter choices that can reach convergence rates of up to 99%. The reliable convergence may make the method a useful tool to study strategic behavior of firms even in more complex settings. Keywords: Bertrand Equilibrium, Competition in Uniform Price Auctions, Deep Deterministic Policy Gradient Algorithm, Parameter Sensitivity Analysis

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑