arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2106.02615cs.GTcs.LG
收藏

共识乘法权重更新:使用基于投影仪的游戏签名学习

Consensus Multiplicative Weights Update: Learning to Learn using Projector-based Game Signatures

J.P. Morgan AI Research(摩根大通人工智能研究院);University of Liverpool(利物浦大学)

Nelson Vadori, Rahul Savani, Thomas Spooner, Sumitra Ganesh

摘要:

Cheung和Piliouras(2020)最近表明,乘法权重更新方法的两个变体- OMWU和MWU -根据游戏是零和还是合作而显示相反的收敛特性。受这项工作和最近的文献学习优化单功能的启发,我们引入了一个新的框架,学习最后一次收敛到纳什均衡的游戏,其中更新规则的系数(学习率)沿轨迹学习的强化学习策略,是有条件的游戏的性质:\textit{the game signature}。我们构造后者使用一个新的分解成8个组件对应于交换投影算子的两个玩家的游戏,推广和统一最近的游戏概念在文献中研究。我们比较了各种更新规则的性能时,他们的系数学习,并表明RL政策是能够利用游戏签名在广泛的游戏类型。在这样做时,我们引入CMWU,一个新的算法,扩展共识优化的约束情况下,具有局部收敛保证零和双矩阵游戏,并显示它享有竞争力的性能与常系数的零和游戏和整个频谱的游戏时,其系数的学习。

英文摘要:

Cheung and Piliouras (2020) recently showed that two variants of the Multiplicative Weights Update method - OMWU and MWU - display opposite convergence properties depending on whether the game is zero-sum or cooperative. Inspired by this work and the recent literature on learning to optimize for single functions, we introduce a new framework for learning last-iterate convergence to Nash Equilibria in games, where the update rule's coefficients (learning rates) along a trajectory are learnt by a reinforcement learning policy that is conditioned on the nature of the game: \textit{the game signature}. We construct the latter using a new decomposition of two-player games into eight components corresponding to commutative projection operators, generalizing and unifying recent game concepts studied in the literature. We compare the performance of various update rules when their coefficients are learnt, and show that the RL policy is able to exploit the game signature across a wide range of game types. In doing so, we introduce CMWU, a new algorithm that extends consensus optimization to the constrained case, has local convergence guarantees for zero-sum bimatrix games, and show that it enjoys competitive performance on both zero-sum games with constant coefficients and across a spectrum of games when its coefficients are learnt.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑