arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2312.08484cs.GT
收藏

自我博弈的Q学习者可以在重复囚徒困境中证明性合谋

Self-Play Q-learners Can Provably Collude in the Iterated Prisoner's Dilemma

Université Jean Monnet Saint-Etienne(圣艾蒂安让莫内大学);CNRS(法国国家科学研究中心);Institut d’Optique Graduate School(高等光学学院);Inria(法国国家信息与自动化研究所);Laboratoire Hubert Curien UMR 5516(于贝尔·居里实验室);Mila(米拉人工智能与智能系统研究所);Université de Montréal(蒙特利尔大学);Università LUISS (Rome)(罗马自由社会科学大学);Toulouse School of Economics(图卢兹经济学院);EIEF(意大利经济基金会);CEPR(欧洲政策研究中心);Canada AI CIFAR Chair(加拿大AI CIFA讲席)

Quentin Bertrand, Juan Duque, Emilio Calvano, Gauthier Gidel

AI总结:

本研究通过理论分析证明自我博弈的Q学习者在重复囚徒困境中能学习合作策略,而非背叛策略,并通过实验验证其鲁棒性。

摘要:

越来越多的计算研究显示,简单的机器学习代理在社会困境中收敛到合作行为,例如寡头市场中的共谋定价,这引发了关于这种结果驱动因素的问题。在本工作中,我们为自我博弈多智能体Q学习者在重复囚徒困境中的这一现象提供了理论基础。我们刻画了在何种广泛条件下,此类代理可以证明性学习合作的帕伏洛(赢留、输转)策略,而不是被帕累托支配的

英文摘要:

A growing body of computational studies shows that simple machine learning agents converge to cooperative behaviors in social dilemmas, such as collusive price-setting in oligopoly markets, raising questions about what drives this outcome. In this work, we provide theoretical foundations for this phenomenon in the context of self-play multi-agent Q-learners in the iterated prisoner's dilemma. We characterize broad conditions under which such agents provably learn the cooperative Pavlov (win-stay, lose-shift) policy rather than the Pareto-dominated "always defect" policy. We validate our theoretical results through additional experiments, demonstrating their robustness across a broader class of deep learning algorithms.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑