Although the field of multi-agent reinforcement learning (MARL) has made considerable progress in the last years, solving systems with a large number of agents remains a hard challenge. Graphon mean field games (GMFGs) enable the scalable analysis of MARL problems that are otherwise intractable. By the mathematical structure of graphons, this approach is limited to dense graphs which are insufficient to describe many real-world networks such as power law graphs. Our paper introduces a novel formulation of GMFGs, called LPGMFGs, which leverages the graph theoretical concept of $L^p$ graphons and provides a machine learning tool to efficiently and accurately approximate solutions for sparse network problems. This especially includes power law networks which are empirically observed in various application areas and cannot be captured by standard graphons. We derive theoretical existence and convergence guarantees and give empirical examples that demonstrate the accuracy of our learning approach for systems with many agents. Furthermore, we extend the Online Mirror Descent (OMD) learning algorithm to our setup to accelerate learning speed, empirically show its capabilities, and conduct a theoretical analysis using the novel concept of smoothed step graphons. In general, we provide a scalable, mathematically well-founded machine learning approach to a large class of otherwise intractable problems of great relevance in numerous research fields.
Using Generative Adversarial Nets on Atari Games for Feature Extraction in Deep Reinforcement Learning
Middle East Technical University(中东技术大学)
深度强化学习(Deep Reinforcement Learning,DRL)已成功应用于机器人导航和自动视频游戏等多个研究领域。然而,这些方法需要过多的计算和与环境的交互,因此需要提高样本效率。这一要求的主要原因是稀疏和延迟的奖励不能为深度神经网络的表示学习提供有效的监督。在这项研究中,近端策略优化(PPO)算法与生成对抗网络(GANs)增强,通过强制网络学习有效的表示而不依赖于稀疏和延迟的奖励作为监督来提高样本效率。结果表明,通过联合训练DRL代理和GAN代理,可以获得更高的性能。- Derin Pekistirmeli Oakme,robot navigasyonu ve otomatikelestirilmis video oyunu oaku a gibi arastirma alanlarinda basariyla uygulanmaktadir.此外,我们还可以看到,由于气候变化和环境变化,我们的工作人员无法确定他们的工作是否正常。但是,在一个人的生活中,人们对自己的生活和生活的看法是,他们的生活和生活是如此的丰富多彩,以至于他们的生活和生活都变得如此的艰难。但据了解,Uretici Cekismeli Aglar(UCA)算法的优化政策方案将使目标明确,并使目标明确或使目标明确,从而使目标明确。其他的声音算法也可以用其他的方法来实现。
Approximate Feedback Nash Equilibria with Sparse Inter-Agent Dependencies
Oden Institute for Computational Engineering and Sciences, The University of Texas at Austin(德克萨斯大学奥斯汀分校奥登计算工程与科学研究所);Department of Electrical Engineering and Computer Sciences, University of California, Berkeley(加州大学伯克利分校电气工程与计算机科学系);DEVCOM Army Research Laboratory(DEVCOM陆军研究实验室)