arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2006.04635cs.LGcs.AIcs.GTcs.MAstat.ML
收藏

学习使用最佳响应政策迭代进行无新闻外交

Learning to Play No-Press Diplomacy with Best Response Policy Iteration

DeepMind
Thomas Anthony, Tom Eccles, Andrea Tacchetti, János Kramár, Ian Gemp, Thomas C. Hudson, Nicolas Porcel, Marc Lanctot, Julien Pérolat, Richard Everett, Roman Wer… 展开作者

Thomas Anthony, Tom Eccles, Andrea Tacchetti, János Kramár, Ian Gemp, Thomas C. Hudson, Nicolas Porcel, Marc Lanctot, Julien Pérolat, Richard Everett, Roman Werpachowski, Satinder Singh, Thore Graepel, Yoram Bachrach

摘要:

深度强化学习(RL)的最新进展已经在许多2人零和游戏中取得了相当大的进展,例如围棋,扑克和星际争霸。这种游戏的纯粹对抗性质允许RL方法在概念上简单且有原则的应用。然而,现实世界的设置是多代理,代理交互是共同利益和竞争方面的复杂混合物。我们认为外交,一个7人的棋盘游戏,旨在强调困境造成的许多代理的相互作用。它还具有较大的组合动作空间和同时移动,这对RL算法来说是一个挑战。我们提出了一个简单而有效的近似最佳反应算子,旨在处理大型组合动作空间和同时移动。我们还介绍了一个家庭的政策迭代方法,近似虚构的发挥。通过这些方法,我们成功地将强化学习应用于外交:我们证明了我们的代理令人信服地优于以前的最先进技术,博弈论均衡分析表明,新的过程产生了一致的改进。

英文摘要:

Recent advances in deep reinforcement learning (RL) have led to considerable progress in many 2-player zero-sum games, such as Go, Poker and Starcraft. The purely adversarial nature of such games allows for conceptually simple and principled application of RL methods. However real-world settings are many-agent, and agent interactions are complex mixtures of common-interest and competitive aspects. We consider Diplomacy, a 7-player board game designed to accentuate dilemmas resulting from many-agent interactions. It also features a large combinatorial action space and simultaneous moves, which are challenging for RL algorithms. We propose a simple yet effective approximate best response operator, designed to handle large combinatorial action spaces and simultaneous moves. We also introduce a family of policy iteration methods that approximate fictitious play. With these methods, we successfully apply RL to Diplomacy: we show that our agents convincingly outperform the previous state-of-the-art, and game theoretic equilibrium analysis shows that the new process yields consistent improvements.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑