多Agent学习中的Poincaré-Bendixson极限集
Poincaré-Bendixson Limit Sets in Multi-Agent Learning
摘要：
演化博弈论和多智能体学习的一个关键挑战是描述博弈动力学的极限行为。而收敛通常是满足特定奖励结构的游戏中的学习算法的特性（例如，零和游戏），甚至基本的学习模型，如复制动态，不能保证收敛一般的回报。更糟糕的是，即使在相当简单的游戏中，如石头剪刀布游戏的变体，也可能出现混乱行为。虽然学习动力学中的混沌行为可以通过著名的庞加莱-本迪克森定理来排除，但它只适用于低维设置。游戏中是否还有其他特征可以迫使学习的极限集合具有规律性？我们表明，与庞加莱-本迪克森定理（极限环，但没有混乱吸引子）一致的行为可以纯粹从交互图的拓扑结构中得出，即使对于具有任意数量的参与者和任意支付矩阵的高维设置也是如此。我们证明了我们的结果，为广泛的一类遵循正则化领导者（FoReL）动态，推广复制动力学，为二元游戏特征的互动图，其中每个玩家的收益只受另一个玩家的影响（即，1的交互图）。由于混沌已经发生在只有两个玩家和三个策略的游戏中，这类非混沌游戏可以被认为是最大的。此外，我们提供了简单的条件下，这种行为转化为效率的保证，这意味着FoReL学习实现的时间平均总收益至少一样好的纳什均衡，从而连接拓扑的动态社会福利分析。
英文摘要：
A key challenge of evolutionary game theory and multi-agent learning is to characterize the limit behavior of game dynamics. Whereas convergence is often a property of learning algorithms in games satisfying a particular reward structure (e.g., zero-sum games), even basic learning models, such as the replicator dynamics, are not guaranteed to converge for general payoffs. Worse yet, chaotic behavior is possible even in rather simple games, such as variants of the Rock-Paper-Scissors game. Although chaotic behavior in learning dynamics can be precluded by the celebrated Poincaré-Bendixson theorem, it is only applicable to low-dimensional settings. Are there other characteristics of a game that can force regularity in the limit sets of learning? We show that behavior consistent with the Poincaré-Bendixson theorem (limit cycles, but no chaotic attractor) can follow purely from the topological structure of the interaction graph, even for high-dimensional settings with an arbitrary number of players and arbitrary payoff matrices. We prove our result for a wide class of follow-the-regularized leader (FoReL) dynamics, which generalize replicator dynamics, for binary games characterized interaction graphs where the payoffs of each player are only affected by one other player (i.e., interaction graphs of indegree one). Since chaos occurs already in games with only two players and three strategies, this class of non-chaotic games may be considered maximal. Moreover, we provide simple conditions under which such behavior translates into efficiency guarantees, implying that FoReL learning achieves time-averaged sum of payoffs at least as good as that of a Nash equilibrium, thereby connecting the topology of the dynamics to social-welfare analysis.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
