均衡策略泛化：一种用于追捕-躲避游戏跨图零样本泛化的强化学习框架
Equilibrium Policy Generalization: A Reinforcement Learning Framework for Cross-Graph Zero-Shot Generalization in Pursuit-Evasion Games
AI总结：
本文提出EPG框架，通过均衡策略训练实现追捕-躲避游戏中跨图零样本泛化的高效解决方案。
摘要：
在对抗性游戏中进行均衡学习是一个重要的课题，广泛受到博弈论和强化学习（RL）领域的研究。追捕-躲避游戏（PEG）作为机器人和安全领域的重要现实游戏类别，需要指数时间才能准确求解。当底层图结构变化时，即使是最先进的RL方法也需要重新计算或至少微调，这可能耗时且影响实时应用。本文提出了一种均衡策略泛化（EPG）框架，以有效学习具有鲁棒跨图零样本性能的通用策略。在PEG的背景下，我们的框架适用于追捕者和躲避者双方在无出口和多出口场景中。这两种泛化特性，据我们所知，是该领域首次出现的。EPG框架的核心思想是在不同的图结构上训练RL策略，以对抗每个单一图的均衡策略。为了构建单图策略的均衡 oracle，我们提出了一种动态规划（DP）算法，可以证明生成纯策略纳什均衡并具有近最优的时间复杂度。为了保证追捕者数量的可扩展性，我们进一步扩展DP和RL，分别设计了分组机制和序列模型用于联合策略分解。实验结果表明，使用均衡指导和为跨图PEG训练提出的距离特征，EPG框架在各种未见过的现实图中保证了有希望的零样本性能。此外，当在具有出口的图上训练时，我们的通用追捕者策略甚至可以匹配最先进PEG方法微调策略的性能。
英文摘要：
Equilibrium learning in adversarial games is an important topic widely examined in the fields of game theory and reinforcement learning (RL). Pursuit-evasion game (PEG), as an important class of real-world games from the fields of robotics and security, requires exponential time to be accurately solved. When the underlying graph structure varies, even the state-of-the-art RL methods require recomputation or at least fine-tuning, which can be time-consuming and impair real-time applicability. This paper proposes an Equilibrium Policy Generalization (EPG) framework to effectively learn a generalized policy with robust cross-graph zero-shot performance. In the context of PEGs, our framework is generally applicable to both pursuer and evader sides in both no-exit and multi-exit scenarios. These two generalizability properties, to our knowledge, are the first to appear in this domain. The core idea of the EPG framework is to train an RL policy across different graph structures against the equilibrium policy for each single graph. To construct an equilibrium oracle for single-graph policies, we present a dynamic programming (DP) algorithm that provably generates pure-strategy Nash equilibrium with near-optimal time complexity. To guarantee scalability with respect to pursuer number, we further extend DP and RL by designing a grouping mechanism and a sequence model for joint policy decomposition, respectively. Experimental results show that, using equilibrium guidance and a distance feature proposed for cross-graph PEG training, the EPG framework guarantees desirable zero-shot performance in various unseen real-world graphs. Besides, when trained under an equilibrium heuristic proposed for the graphs with exits, our generalized pursuer policy can even match the performance of the fine-tuned policies from the state-of-the-art PEG methods.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
