arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2407.20128math.OCcs.GTstat.ML
收藏

零和多矩阵博弈中学习动态的有限样本保证

Finite-Sample Guarantees for Learning Dynamics in Zero-Sum Polymatrix Games

Fathima Zarin Faizal, Asuman Ozdaglar, Martin J. Wainwright

AI总结:

研究了零和多矩阵博弈在完全信息和最小信息设定下的最佳响应型学习动态,提出双时间尺度动态结合平滑最佳响应与TD-learning,并证明了在无额外探索下收敛至$ε$-Nash均衡的多项式时间有限样本保证。

摘要:

我们研究了两种信息设定下零和多矩阵博弈的最佳响应型学习动态。这两种设定的区别在于每个玩家对博弈及其对手策略所掌握的信息类型。第一种设定是完全信息情形,每个玩家知道自己和对手的收益矩阵,并观察所有人的混合策略。第二种设定是最小信息情形,玩家不观察对手的策略,也不了解任何收益矩阵(他们仅观察自己的已实现收益)。对于这种在学习博弈文献中也被称为彻底解耦的情形,我们研究了一种双时间尺度学习动态,它将用于策略估计的平滑最佳响应型更新与用于估计局部收益函数的 TD-learning 更新相结合。对于这些动态,在没有额外探索的情况下,我们提供了收敛至 $ε$-Nash 均衡的多项式时间有限样本保证。

英文摘要:

We study best-response type learning dynamics for zero-sum polymatrix games under two information settings. The two settings are distinguished by the type of information that each player has about the game and their opponents' strategy. The first setting is the full information case, in which each player knows their own and their opponents' payoff matrices and observes everyone's mixed strategies. The second setting is the minimal information case, where players do not observe their opponents' strategies and are not aware of any payoff matrices (instead they only observe their realized payoffs). For this setting, also known as the radically uncoupled case in the learning in games literature, we study a two-timescale learning dynamics that combine smoothed best-response type updates for strategy estimates with a TD-learning update to estimate a local payoff function. For these dynamics, without additional exploration, we provide polynomial-time finite-sample guarantees for convergence to an $ε$-Nash equilibrium.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑