求解群体间的双人一般和博弈
Solving Two-Player General-Sum Games Between Swarms
AI总结:
针对高维下HJI偏微分方程求解双人一般和博弈的维数灾难问题,将研究拓展至双人群体博弈,用柯尔莫哥洛夫前向方程建模群体演化,基于PINN生成的策略收益优于Nash DDQN、与数值求解器相当。
摘要:
Hamilton-Jacobi-Isaacs(HJI,哈密顿-雅可比-艾萨克斯)偏微分方程(PDEs)是双人一般和博弈的控制方程。与依赖大量数据学习价值函数的强化学习(RL)方法不同,学习HJ偏微分方程能在博弈纳什均衡值存在时保证收敛到该值。但一个局限是,当状态维度升高时,求解HJ偏微分方程会变得难以处理。为规避维数灾难(CoD),可采用带监督的物理启发机器学习方法,这类方法已被证明能有效生成双人一般和博弈的均衡策略。本研究将现有的智能体层级双人博弈研究拓展至双人群体层级博弈,即两个子群体进行一般和博弈。我们采用柯尔莫哥洛夫前向方程(Kolmogorov forward equation)作为群体密度演化的动力学模型。结果表明,由物理启发神经网络(PINN)生成的策略收益高于纳什双深度Q网络(Nash DDQN)智能体,且性能与数值求解器相当。
英文摘要:
Hamilton-Jacobi-Isaacs (HJI) PDEs are the governing equations for the two-player general-sum games. Unlike Reinforcement Learning (RL) methods, which are data-intensive methods for learning value function, learning HJ PDEs provide a guaranteed convergence to the Nash Equilibrium value of the game when it exists. However, a caveat is that solving HJ PDEs becomes intractable when the state dimension increases. To circumvent the curse of dimensionality (CoD), physics-informed machine learning methods with supervision can be used and have been shown to be effective in generating equilibrial policies in two-player general-sum games. In this work, we extend the existing work on agent-level two-player games to a two-player swarm-level game, where two sub-swarms play a general-sum game. We consider the \textit{Kolmogorov forward equation} as the dynamic model for the evolution of the densities of the swarms. Results show that policies generated from the physics-informed neural network (PINN) result in a higher payoff than a Nash Double Deep Q-Network (Nash DDQN) agent and have comparable performance with numerical solvers.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。