斗地主的组合Q学习
Combinational Q-Learning for Dou Di Zhu
摘要：
深度强化学习（DRL）近年来获得了广泛的关注，并已被证明能够在人类水平或更高的水平上玩雅达利游戏和围棋。然而，这些游戏被假设为具有少量固定数量的动作，并且可以使用简单的CNN网络进行训练。在本文中，我们研究了一类特殊的亚洲流行的纸牌游戏，称为斗地主，其中两个敌对的群体的代理必须考虑许多卡组合在每个时间步，导致大量的行动。我们提出了一种新的方法来处理组合动作，我们称之为组合Q学习（CQL）。我们采用两阶段网络来减少动作空间，并利用顺序不变的最大池操作来提取原始动作之间的关系。结果表明，我们的方法优于最先进的方法，如朴素Q学习和A3C。我们开发了一个易于使用的纸牌游戏环境和训练所有代理人adversarially从sractch，只有游戏规则的知识，并验证我们的代理人是比较人类。我们的代码，以重现所有报告的结果将在网上提供。
英文摘要：
Deep reinforcement learning (DRL) has gained a lot of attention in recent years, and has been proven to be able to play Atari games and Go at or above human levels. However, those games are assumed to have a small fixed number of actions and could be trained with a simple CNN network. In this paper, we study a special class of Asian popular card games called Dou Di Zhu, in which two adversarial groups of agents must consider numerous card combinations at each time step, leading to huge number of actions. We propose a novel method to handle combinatorial actions, which we call combinational Q-learning (CQL). We employ a two-stage network to reduce action space and also leverage order-invariant max-pooling operations to extract relationships between primitive actions. Results show that our method prevails over state-of-the art methods like naive Q-learning and A3C. We develop an easy-to-use card game environments and train all agents adversarially from sractch, with only knowledge of game rules and verify that our agents are comparative to humans. Our code to reproduce all reported results will be available online.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
