扩展形式粗相关均衡的有效分散学习动态：无需昂贵的平稳分布计算
Efficient Decentralized Learning Dynamics for Extensive-Form Coarse Correlated Equilibrium: No Expensive Computation of Stationary Distributions Required
摘要：
虽然在两人零和博弈中，纳什均衡是一个公认的最优博弈的规定性概念，但它作为规定性工具的适用性是有限的。因此，分散学习动态的研究，保证收敛到相关的解决方案的概念，在多层，一般和扩展形式（即，树形）游戏已经成为活跃研究的重要课题。目前已知的学习动态的每次迭代的复杂性取决于所考虑的特定相关的解决方案的概念。例如，在扩展形式相关均衡（EFCE）的情况下，所有已知的动力学都需要在每次迭代的中间步骤中计算多个马尔可夫链的平稳分布，这在实践中是一个昂贵的操作。opponent，在正常形式的粗相关均衡（NFCCE）的情况下，简单的无外部遗憾学习动态，相当于每个代理的树形决策空间的线性时间遍历足以保证收敛。本文重点研究了扩展形式粗相关均衡（EFCCE），它是NFCCE的一个子集和EFCE的一个超集。作为EFCE的超集，EFCE的任何学习动态自动保证收敛到EFCCE。然而，由于EFCCE是一个更简单的解决方案概念，这就引出了一个问题：EFCCE的学习动力学是否存在，以避免昂贵的静态分布计算？本文从肯定的角度回答了上述问题。我们的学习动态只需要编排无外部遗憾最小化，从而表明，从学习的角度来看，EFCCE更类似于NFCCE而不是EFCE。我们的动力学保证了$T$迭代后的经验播放频率是一个$O(1/\sqrt{T})$-近似EFCCE具有高概率，和EFCCE几乎肯定在极限。
英文摘要：
While in two-player zero-sum games the Nash equilibrium is a well-established prescriptive notion of optimal play, its applicability as a prescriptive tool beyond that setting is limited. Consequently, the study of decentralized learning dynamics that guarantee convergence to correlated solution concepts in multiplayer, general-sum extensive-form (i.e., tree-form) games has become an important topic of active research. The per-iteration complexity of the currently known learning dynamics depends on the specific correlated solution concept considered. For example, in the case of extensive-form correlated equilibrium (EFCE), all known dynamics require, as an intermediate step at each iteration, to compute the stationary distribution of multiple Markov chains, an expensive operation in practice. Oppositely, in the case of normal-form coarse correlated equilibrium (NFCCE), simple no-external-regret learning dynamics that amount to a linear-time traversal of the tree-form decision space of each agent suffice to guarantee convergence. This paper focuses on extensive-form coarse correlated equilibrium (EFCCE), an intermediate solution concept that is a subset of NFCCE and a superset of EFCE. Being a superset of EFCE, any learning dynamics for EFCE automatically guarantees convergence to EFCCE. However, since EFCCE is a simpler solution concept, this begs the question: do learning dynamics for EFCCE that avoid the expensive computation of stationary distributions exist? This paper answers the previous question in the positive. Our learning dynamics only require the orchestration of no-external-regret minimizers, thus showing that EFCCE is more akin to NFCCE than to EFCE from a learning perspective. Our dynamics guarantees that the empirical frequency of play after $T$ iteration is a $O(1/\sqrt{T})$-approximate EFCCE with high probability, and an EFCCE almost surely in the limit.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
