学习使用最佳响应政策迭代进行无新闻外交
Learning to Play No-Press Diplomacy with Best Response Policy Iteration
摘要:
深度强化学习(RL)的最新进展已经在许多2人零和游戏中取得了相当大的进展,例如围棋,扑克和星际争霸。这种游戏的纯粹对抗性质允许RL方法在概念上简单且有原则的应用。然而,现实世界的设置是多代理,代理交互是共同利益和竞争方面的复杂混合物。我们认为外交,一个7人的棋盘游戏,旨在强调困境造成的许多代理的相互作用。它还具有较大的组合动作空间和同时移动,这对RL算法来说是一个挑战。我们提出了一个简单而有效的近似最佳反应算子,旨在处理大型组合动作空间和同时移动。我们还介绍了一个家庭的政策迭代方法,近似虚构的发挥。通过这些方法,我们成功地将强化学习应用于外交:我们证明了我们的代理令人信服地优于以前的最先进技术,博弈论均衡分析表明,新的过程产生了一致的改进。
英文摘要:
Recent advances in deep reinforcement learning (RL) have led to considerable progress in many 2-player zero-sum games, such as Go, Poker and Starcraft. The purely adversarial nature of such games allows for conceptually simple and principled application of RL methods. However real-world settings are many-agent, and agent interactions are complex mixtures of common-interest and competitive aspects. We consider Diplomacy, a 7-player board game designed to accentuate dilemmas resulting from many-agent interactions. It also features a large combinatorial action space and simultaneous moves, which are challenging for RL algorithms. We propose a simple yet effective approximate best response operator, designed to handle large combinatorial action spaces and simultaneous moves. We also introduce a family of policy iteration methods that approximate fictitious play. With these methods, we successfully apply RL to Diplomacy: we show that our agents convincingly outperform the previous state-of-the-art, and game theoretic equilibrium analysis shows that the new process yields consistent improvements.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。