无记忆算法合谋:必然失败,衰落缓慢
Memoryless Algorithmic Collusion: Sure to Fail, Slow to Fall
AI总结:
本文针对伯特兰式竞争博弈,研究无记忆Q学习算法的收敛特性,提出结构模型解释折扣因子作用并揭示学习率非单调关系,发现高折扣因子下收敛时间超指数增长。
摘要:
本文表明,在一类伯特兰式竞争博弈中,无记忆Q学习算法经长期充分探索后应收敛至纳什均衡,加速模拟验证了该结论,但高折扣因子下收敛时间呈超指数增长。合谋结果在短期的韧性源于对无利可图行动的探索,使系统类似随机游走。本文提出结构模型估计收敛时间,不仅解释了折扣因子的作用,还揭示了学习率的非单调关系,这是现有文献忽略的。
英文摘要:
This paper shows that, in a class of Bertrand-style competition games, memoryless Q-learning algorithms should adapt to Nash Equilibrium given sufficient explorations in the long-run. This is also verified through accelerating simulations, while the convergence time grows super-exponentially for high discount factors. The resilience of collusive outcomes in the short-run is due to exploration of unprofitable actions, making the system resemble random walk. A structural model is proposed to estimate the convergence time, which not only explains the role of discount factor, but also unveils the non-monotonic relation in learning rate, which is overlooked in the literature.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。