arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2002.09806math.OCcs.GTstat.ML
收藏

多智能体博弈学习的最短时间最后迭代收敛性

Finite-Time Last-Iterate Convergence for Multi-Agent Learning in Games

Tianyi Lin, Zhengyuan Zhou, Panayotis Mertikopoulos, Michael I. Jordan

摘要:

在本文中,我们认为多智能体学习通过在线梯度下降一类游戏称为$λ$ -cocoercive游戏,一个相当广泛的游戏,承认许多纳什均衡,并适当地包括无约束强单调游戏。我们刻画了$λ$ -余度博弈上联合OGD学习的有限时间最后收敛速度;在此基础上,我们提出了一种完全自适应的OGD学习算法,该算法不需要任何问题参数的知识(例如协迫常数$λ$),并通过新的双停时技术显示,该自适应算法与非自适应算法具有相同的有限时间收敛速度收敛速度。随后,我们将OGD学习扩展到噪声梯度反馈的情况下,并建立最后的收敛结果-第一定性的几乎必然收敛,然后定量的有限时间收敛速度-所有在非递减步长。据我们所知,我们提供了第一组结果,填补了现有的多智能体在线学习文献中的几个空白,其中三个方面-有限时间收敛速度,非递减步长和完全自适应算法之前一直未被探索。

英文摘要:

In this paper, we consider multi-agent learning via online gradient descent in a class of games called $λ$-cocoercive games, a fairly broad class of games that admits many Nash equilibria and that properly includes unconstrained strongly monotone games. We characterize the finite-time last-iterate convergence rate for joint OGD learning on $λ$-cocoercive games; further, building on this result, we develop a fully adaptive OGD learning algorithm that does not require any knowledge of problem parameter (e.g. cocoercive constant $λ$) and show, via a novel double-stopping time technique, that this adaptive algorithm achieves same finite-time last-iterate convergence rate as non-adaptive counterpart. Subsequently, we extend OGD learning to the noisy gradient feedback case and establish last-iterate convergence results -- first qualitative almost sure convergence, then quantitative finite-time convergence rates -- all under non-decreasing step-sizes. To our knowledge, we provide the first set of results that fill in several gaps of the existing multi-agent online learning literature, where three aspects -- finite-time convergence rates, non-decreasing step-sizes, and fully adaptive algorithms have been unexplored before.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑