arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 1805.02777cs.LGcs.GTcs.MAstat.ML
收藏

我们在玩什么游戏?在正常和扩展形式的游戏中进行端到端学习

What game are we playing? End-to-end learning in normal and extensive form games

Carnegie Mellon University(卡内基梅隆大学)

Chun Kai Ling, Fei Fang, J. Zico Kolter

摘要:

虽然最近的人工智能工作在解决大型、零和、可拓形式的游戏方面取得了很大进展,但过去大多数工作的基本假设是,游戏本身的参数是智能体已知的。本文论述了相对未被充分探索但同样重要的“逆”设置,其中底层游戏的参数并不为所有代理所知,但必须通过观察来学习。我们提出了一个差异化的,端到端的学习框架来解决这个任务。特别是,我们考虑了一个正则化版本的游戏,相当于一个特定形式的量子响应平衡,并开发1)原始对偶牛顿方法,在正常和广泛的形式的游戏中找到这样的平衡点;和2)反向传播方法,让我们分析计算梯度的所有相关的游戏参数通过解决方案本身。这最终让我们通过端到端的方式学习游戏,有效地将“可微游戏求解器”集成到更大的深度网络架构的循环中。我们证明了学习方法的有效性,在几个设置,包括扑克和安全游戏任务。

英文摘要:

Although recent work in AI has made great progress in solving large, zero-sum, extensive-form games, the underlying assumption in most past work is that the parameters of the game itself are known to the agents. This paper deals with the relatively under-explored but equally important "inverse" setting, where the parameters of the underlying game are not known to all agents, but must be learned through observations. We propose a differentiable, end-to-end learning framework for addressing this task. In particular, we consider a regularized version of the game, equivalent to a particular form of quantal response equilibrium, and develop 1) a primal-dual Newton method for finding such equilibrium points in both normal and extensive form games; and 2) a backpropagation method that lets us analytically compute gradients of all relevant game parameters through the solution itself. This ultimately lets us learn the game by training in an end-to-end fashion, effectively by integrating a "differentiable game solver" into the loop of larger deep network architectures. We demonstrate the effectiveness of the learning method in several settings including poker and security game tasks.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑