NeuPL:神经群体学习
NeuPL: Neural Population Learning
摘要:
在策略游戏(如星际争霸,扑克)中学习需要发现不同的策略。这通常是通过针对现有策略迭代地训练新策略来实现的,从而增加了一个健壮的策略种群。这种迭代方法在现实世界的游戏中存在两个问题:a)在有限的预算下,每次迭代的近似最佳反应算子需要截断,导致训练不足的良好反应填充种群; b)在每次迭代中重复学习基本技能是浪费的,并且在越来越强大的对手面前变得棘手。在这项工作中,我们提出了神经群体学习(NeuPL)作为这两个问题的解决方案。NeuPL在温和的假设下为最佳反应群体提供了收敛保证。通过在单个条件模型中表示策略群体,NeuPL支持跨策略的迁移学习。从经验上讲,我们展示了NeuPL在几个测试领域的通用性,提高了性能和效率。最有趣的是,我们发现,随着神经元数量的增加,新的策略变得更容易获得,而不是更少。
英文摘要:
Learning in strategy games (e.g. StarCraft, poker) requires the discovery of diverse policies. This is often achieved by iteratively training new policies against existing ones, growing a policy population that is robust to exploit. This iterative approach suffers from two issues in real-world games: a) under finite budget, approximate best-response operators at each iteration needs truncating, resulting in under-trained good-responses populating the population; b) repeated learning of basic skills at each iteration is wasteful and becomes intractable in the presence of increasingly strong opponents. In this work, we propose Neural Population Learning (NeuPL) as a solution to both issues. NeuPL offers convergence guarantees to a population of best-responses under mild assumptions. By representing a population of policies within a single conditional model, NeuPL enables transfer learning across policies. Empirically, we show the generality, improved performance and efficiency of NeuPL across several test domains. Most interestingly, we show that novel strategies become more accessible, not less, as the neural population expands.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。