Nowadays, a significant share of the Business-to-Consumer sector is based on online platforms like Amazon and Alibaba and uses Artificial Intelligence for pricing strategies. This has sparked debate on whether pricing algorithms may tacitly collude to set supra-competitive prices without being explicitly designed to do so. Our study addresses these concerns by examining the risk of collusion when Reinforcement Learning algorithms are used to decide on pricing strategies in competitive markets. Prior research in this field focused on Tabular Q-learning (TQL) and led to opposing views on whether learning-based algorithms can lead to supra-competitive prices. Our work contributes to this ongoing discussion by providing a more nuanced numerical study that goes beyond TQL by additionally capturing off- and on-policy Deep Reinforcement Learning (DRL) algorithms. We study multiple Bertrand oligopoly variants and show that algorithmic collusion depends on the algorithm used. In our experiments, TQL exhibits higher collusion and price dispersion phenomena compared to DRL algorithms. We show that the severity of collusion depends not only on the algorithm used but also on the characteristics of the market environment. We further find that Proximal Policy Optimization appears to be less sensitive to collusive outcomes compared to other state-of-the-art DRL algorithms.
a pawfessor of economics and of computer science(经济与计算机科学教授);OpenAI’s Researcher Access Program(OpenAI研究员访问计划);Google’s Gemini Academic Program(Google的Gemini学术计划);Cloud Research Credits Program(云研究信用计划);Anthropic;NSF Graduate Research Fellowship(NSF研究生研究 fellowship);Kempner Institute Graduate Fellowship(Kempner研究所研究生 fellowship);National Science Foundation (NSF-BSF grant No. 2343922)(国家科学基金会(NSF-BSF grant No. 2343922));Harvard FAS Dean’s Competitive Fund for Promising Scholarship(哈佛大学哈佛大学教务处有前途的学术研究竞争基金);Harvard FAS Inequality in America Initiative(哈佛大学哈佛大学美国不平等倡议);United States–Israel Binational Science Foundation (BSF grant 2022417)(美国-以色列双边科学基金会(BSF grant 2022417))
The Spoils of Algorithmic Collusion: Profit Allocation Among Asymmetric Firms
University of Vienna(维也纳大学);Heinrich-Heine-Universität Düsseldorf(杜塞尔多夫海因里希·海涅大学);Max Planck Institute for Research on Collective Goods(马克斯·普朗克集体利益研究所);E.CA Economics GmbH(E.CA经济咨询公司);Max Planck Institute for Human Development(马克斯·普朗克人类发展研究所)