arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2402.07355math.STcs.LGstat.MLstat.TH
收藏

从平均场稳态分布中采样

Sampling from the Mean-Field Stationary Distribution

Institute for Advanced Study(高等研究院);University of Toronto(多伦多大学);Vector Institute(矢量研究所);Princeton University(普林斯顿大学);Georgia Institute of Technology(佐治亚理工学院)

Yunbum Kook, Matthew S. Zhang, Sinho Chewi, Murat A. Erdogdu, Mufan Bill Li

AI总结:

本文通过解耦有限粒子近似与对数凹采样,提出从平均场SDE稳态分布采样的新方法,并建立关于粒子数一致的对数Sobolev不等式,改进了平均场情形下两层神经网络优化等任务的保证。

摘要:

我们研究从平均场随机微分方程(SDE)的稳态分布中采样的复杂度,等价地,研究在包含交互项的概率测度空间上最小化一个泛函的复杂度。我们的核心见解是将该问题的两个关键方面解耦:(1)通过有限粒子系统近似平均场 SDE,借助时间一致的混沌传播;(2)通过标准对数凹采样器从有限粒子稳态分布中采样。我们的方法在概念上更简单,其灵活性允许同时融入算法和理论方面的最新成果。这在众多场景下带来了改进的保证,包括在平均场机制下优化某些两层神经网络的更优保证。一项关键的技术贡献是为平均场 Langevin 动力学的稳态分布建立了一个新的关于 $N$ 一致的对数 Sobolev 不等式。

英文摘要:

We study the complexity of sampling from the stationary distribution of a mean-field SDE, or equivalently, the complexity of minimizing a functional over the space of probability measures which includes an interaction term. Our main insight is to decouple the two key aspects of this problem: (1) approximation of the mean-field SDE via a finite-particle system, via uniform-in-time propagation of chaos, and (2) sampling from the finite-particle stationary distribution, via standard log-concave samplers. Our approach is conceptually simpler and its flexibility allows for incorporating the state-of-the-art for both algorithms and theory. This leads to improved guarantees in numerous settings, including better guarantees for optimizing certain two-layer neural networks in the mean-field regime. A key technical contribution is to establish a new uniform-in-$N$ log-Sobolev inequality for the stationary distribution of the mean-field Langevin dynamics.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑