arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2209.09893stat.MLcs.GTcs.LGmath.OC
收藏

深广义薛定谔桥

Deep Generalized Schrödinger Bridge

Georgia Institute of Technology(佐治亚理工学院);Massachusetts Institute of Technology(麻省理工学院)

Guan-Horng Liu, Tianrong Chen, Oswin So, Evangelos A. Theodorou

摘要:

平均场博弈(MFG)是一个重要的数学框架,用于模拟个体与大种群随机交互的集体行为。在这项工作中,我们的目标是解决一类具有挑战性的MFG,其中这些相互作用的偏好的可微性可能无法提供给求解器,并敦促人口精确地收敛到一些所需的分布。这些设置,尽管出于实用目的是很好的动机,复杂到足以瘫痪大多数(深度)数值求解器。然而,我们表明,薛定谔桥-作为一个熵正则化的最佳传输模型-可以推广到接受平均场结构,从而解决这些MFG。这是通过应用前向-后向随机微分方程理论来实现的,有趣的是,这导致了一个与时间差学习具有类似结构的计算框架。因此,它为深度强化学习开辟了新的算法连接,我们可以利用它来促进实际训练。我们表明,我们提出的目标函数提供了必要和充分条件的平均场问题。我们的方法,命名为深广义薛定谔桥(DeepGSB),不仅优于以前的方法在解决经典的人口导航MFG,但也能够解决1000维意见去极化,设置一个新的国家的最先进的数值求解器的高维MFG。我们的代码将在https://github.com/ghliu/DeepGSB上提供。

英文摘要:

Mean-Field Game (MFG) serves as a crucial mathematical framework in modeling the collective behavior of individual agents interacting stochastically with a large population. In this work, we aim at solving a challenging class of MFGs in which the differentiability of these interacting preferences may not be available to the solver, and the population is urged to converge exactly to some desired distribution. These setups are, despite being well-motivated for practical purposes, complicated enough to paralyze most (deep) numerical solvers. Nevertheless, we show that Schrödinger Bridge - as an entropy-regularized optimal transport model - can be generalized to accepting mean-field structures, hence solving these MFGs. This is achieved via the application of Forward-Backward Stochastic Differential Equations theory, which, intriguingly, leads to a computational framework with a similar structure to Temporal Difference learning. As such, it opens up novel algorithmic connections to Deep Reinforcement Learning that we leverage to facilitate practical training. We show that our proposed objective function provides necessary and sufficient conditions to the mean-field problem. Our method, named Deep Generalized Schrödinger Bridge (DeepGSB), not only outperforms prior methods in solving classical population navigation MFGs, but is also capable of solving 1000-dimensional opinion depolarization, setting a new state-of-the-art numerical solver for high-dimensional MFGs. Our code will be made available at https://github.com/ghliu/DeepGSB.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑