多重记忆博弈中的学习引发偏离纳什均衡的复杂动力学
Learning in Multi-Memory Games Triggers Complex Dynamics Diverging from Nash Equilibrium
AI总结：
本研究将复制器动力学和梯度上升扩展至多重记忆博弈，证明两者动力学相同，并发现其在零和博弈中偏离纳什均衡形成异宿循环。
摘要：
重复博弈考虑多个智能体在整个学习过程中受各自独立奖励驱动的情境。通常，它们的学习动力学会变得复杂。特别是当它们的奖励相互竞争时，如零和博弈，动力学往往无法收敛到其最优解，即纳什均衡。为了应对这种复杂性，许多研究将各种学习算法理解为动力系统，并发现了算法间的定性见解。然而，此类研究尚未处理多重记忆博弈（即智能体能够记住过去执行的动作，并基于记忆选择动作），尽管记忆在人工智能和人际关系中起着关键作用。本研究将博弈中的两种主要学习算法，即复制器动力学和梯度上升，扩展到多重记忆博弈中。然后，我们证明了它们的动力学是相同的。此外，在理论和实验上，我们阐明了在多重记忆零和博弈中，学习动力学会偏离纳什均衡并到达异宿循环（在策略空间边界附近逗留更长时间），为博弈学习提供了基础性进展。
英文摘要：
Repeated games consider a situation where multiple agents are motivated by their independent rewards throughout learning. In general, the dynamics of their learning become complex. Especially when their rewards compete with each other like zero-sum games, the dynamics often do not converge to their optimum, i.e., the Nash equilibrium. To tackle such complexity, many studies have understood various learning algorithms as dynamical systems and discovered qualitative insights among the algorithms. However, such studies have yet to handle multi-memory games (where agents can memorize actions they played in the past and choose their actions based on their memories), even though memorization plays a pivotal role in artificial intelligence and interpersonal relationship. This study extends two major learning algorithms in games, i.e., replicator dynamics and gradient ascent, into multi-memory games. Then, we prove their dynamics are identical. Furthermore, theoretically and experimentally, we clarify that the learning dynamics diverge from the Nash equilibrium in multi-memory zero-sum games and reach heteroclinic cycles (sojourn longer around the boundary of the strategy space), providing a fundamental advance in learning in games.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
