通过利用2p0s微分博弈的均衡结构来解决足球
Solving Football by Exploiting Equilibrium Structure of 2p0s Differential Games with One-Sided Information
AI总结：
通过利用2p0s微分博弈的均衡结构，解决具有连续动作空间的22人足球游戏，提升学习准确性和效率。
摘要：
对于一个两位玩家不完全信息扩展型博弈（IIEFG）具有K个时间步和玩家动作空间大小为U，游戏树复杂度为U^{2K}，导致现有IIEFG求解器在处理大或无限（U,K）时遇到困难，例如具有连续动作空间的微分博弈。为了部分解决这个可扩展性挑战，我们专注于一个重要的2p0s类游戏，其中知情玩家（P1）知道收益，而不知情玩家（P2）只对I个可能的收益集合有信念。此类游戏涵盖了体育、防御、网络安全和金融等领域的广泛场景。我们证明，在温和的条件下，P1（或P2）在任何信息状态下的均衡策略集中在最多I（或I+1）个动作原型上。当I<<U时，这种均衡结构导致当P2采取最佳反应时，P1的游戏树复杂度降至I^K，而在P1采取最佳反应的双人游戏中，P2的复杂度降至（I+1）^K。我们随后展示，利用这种结构在无模型多智能体强化学习和模型预测控制中，能够显著提高从SOTA IIEFG求解器中学习的准确性和效率。我们的演示解决了具有连续动作空间和K=10时间步的22人足球游戏，其中进攻方需要在关键时刻战略性地隐藏他们的战术以利用信息优势。代码可在https://github.com/ghimiremukesh/cams/tree/iclr获取。
英文摘要：
For a two-player imperfect-information extensive-form game (IIEFG) with $K$ time steps and a player action space of size $U$, the game tree complexity is $U^{2K}$, causing existing IIEFG solvers to struggle with large or infinite $(U,K)$, e.g., differential games with continuous action spaces. To partially address this scalability challenge, we focus on an important class of 2p0s games where the informed player (P1) knows the payoff while the uninformed player (P2) only has a belief over the set of $I$ possible payoffs. Such games encompass a wide range of scenarios in sports, defense, cybersecurity, and finance. We prove that under mild conditions, P1's (resp. P2's) equilibrium strategy at any infostate concentrates on at most $I$ (resp. $I+1$) action prototypes. When $I\ll U$, this equilibrium structure causes the game tree complexity to collapse to $I^K$ for P1 when P2 plays best responses, and $(I+1)^K$ for P2 in a dual game where P1 plays best responses. We then show that exploiting this structure in model-free multiagent reinforcement learning and model predictive control leads to significant improvements in learning accuracy and efficiency from SOTA IIEFG solvers. Our demonstration solves a 22-player football game with continuous action spaces and $K=10$ time steps, where the offense team needs to strategically conceal their play until a critical moment in order to exploit information advantage. Code is available at https://github.com/ghimiremukesh/cams/tree/iclr
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
