连续对策Helmholtz分解的牛顿优化
Newton Optimization on Helmholtz Decomposition for Continuous Games
摘要：
许多学习问题涉及多个代理优化不同的交互功能。在这些问题中，标准的策略梯度算法由于设置的非平稳性和每个代理的不同利益而失败。事实上，算法必须考虑到这些系统的复杂动态，以保证快速收敛到（局部）纳什均衡。在本文中，我们提出了NOHD（牛顿优化亥姆霍兹分解），牛顿式算法的多智能体学习问题的基础上分解的系统的动力学在其无旋（潜在的）和螺线管（哈密顿）组件。该方法保证了纯无旋系统和纯螺线管系统的二次收敛性。此外，我们表明，NOHD被吸引到稳定的不动点在一般的多智能体系统和排斥严格的鞍。最后，我们经验比较的NOHD的性能与国家的最先进的算法在一些双矩阵游戏，并在一个连续的Gridworld环境。
英文摘要：
Many learning problems involve multiple agents optimizing different interactive functions. In these problems, the standard policy gradient algorithms fail due to the non-stationarity of the setting and the different interests of each agent. In fact, algorithms must take into account the complex dynamics of these systems to guarantee rapid convergence towards a (local) Nash equilibrium. In this paper, we propose NOHD (Newton Optimization on Helmholtz Decomposition), a Newton-like algorithm for multi-agent learning problems based on the decomposition of the dynamics of the system in its irrotational (Potential) and solenoidal (Hamiltonian) component. This method ensures quadratic convergence in purely irrotational systems and pure solenoidal systems. Furthermore, we show that NOHD is attracted to stable fixed points in general multi-agent systems and repelled by strict saddle ones. Finally, we empirically compare the NOHD's performance with that of state-of-the-art algorithms on some bimatrix games and in a continuous Gridworld environment.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
