arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 1905.04532cs.GTcs.LGcs.MA
收藏

零和游戏中的快速和激烈学习：以非消失步长消除遗憾

Fast and Furious Learning in Zero-Sum Games: Vanishing Regret with Non-Vanishing Step Sizes

Singapore University of Technology and Design（新加坡科技设计大学）

James P. Bailey, Georgios Piliouras

阅读 PDF
arXiv 页面 ↗HTML 原文 ↗

摘要：

据我们所知，我们第一次证明，在零和游戏的在线学习中，有可能调和两个看似矛盾的目标：消失的时间平均后悔和非消失的步长。这种现象，我们在游戏中创造了“快速而激烈”的学习，为最大最小优化和多智能体系统的可能性设定了一个新的基准。我们的分析并不依赖于引入一个精心定制的动态。相反，我们专注于最好的研究在线动态，梯度下降。同样，我们专注于最简单的教科书游戏类，两个代理两个策略零和游戏，如匹配便士。即使对于这个最简单的基准测试，在我们的工作之前，最著名的完全后悔的界限是$O(T)$的平凡界限，它甚至可以直接应用于非学习代理。基于对对偶空间中非平衡轨迹几何的严格理解，我们证明了$Θ(\sqrt{T})$的遗憾界与在线设置中自适应步长的众所周知的最佳界相匹配。这一保证适用于所有固定步长，而不必提前知道时间范围并相应地调整固定步长。作为一个推论，我们建立，即使有固定的学习率的时间平均的混合策略，效用收敛到其确切的纳什均衡值。

英文摘要：

We show for the first time, to our knowledge, that it is possible to reconcile in online learning in zero-sum games two seemingly contradictory objectives: vanishing time-average regret and non-vanishing step sizes. This phenomenon, that we coin ``fast and furious" learning in games, sets a new benchmark about what is possible both in max-min optimization as well as in multi-agent systems. Our analysis does not depend on introducing a carefully tailored dynamic. Instead we focus on the most well studied online dynamic, gradient descent. Similarly, we focus on the simplest textbook class of games, two-agent two-strategy zero-sum games, such as Matching Pennies. Even for this simplest of benchmarks the best known bound for total regret, prior to our work, was the trivial one of $O(T)$, which is immediately applicable even to a non-learning agent. Based on a tight understanding of the geometry of the non-equilibrating trajectories in the dual space we prove a regret bound of $Θ(\sqrt{T})$ matching the well known optimal bound for adaptive step sizes in the online setting. This guarantee holds for all fixed step-sizes without having to know the time horizon in advance and adapt the fixed step-size accordingly. As a corollary, we establish that even with fixed learning rates the time-average of mixed strategies, utilities converge to their exact Nash equilibrium values.

机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。

↑