arXiv 1909.055032019-09-12cs.LGcs.DSmath.PRstat.ML

对数凹抽样的随机中点法

The Randomized Midpoint Method for Log-Concave Sampling

University of Washington（华盛顿大学）；Microsoft Research（微软研究院）

Ruoqi Shen, Yin Tat Lee

摘要：

对数凹分布的抽样是一个在统计学和机器学习中有许多应用的研究问题。本文研究了$p^{*}\propto\exp(-f(x))$型的分布，其中$f:\mathbb{R}^{d}\rightarrow\mathbb{R}$具有$L$ -Lipschitz梯度，并且是$m$ -强凸的.本文提出了一种基于欠阻尼朗之万扩散（ULD）的马尔可夫链蒙特卡罗（MCMC）算法。它可以在$\tilde{O}\left(κ^{7/6}/ε^{1/3}+κ/ε^{2/3}\right)$步骤中实现$ε\cdot D$误差（以2-Wasserstein距离计），其中$D\overset{\mathrm{def}}{=}\sqrt{\frac{d}{m}}$是问题的有效直径，$κ\overset{\mathrm{def}}{=}\frac{L}{m}$是条件数。我们的算法执行速度显着比以前最好的已知算法解决这个问题，这需要$\tilde{O}\left(κ^{1.5}/ε\right)$的步骤。此外，我们的算法可以很容易地并行化，只需要$O(κ\log\frac{1}ε)$并行步骤。为了解决抽样问题，我们提出了一个新的框架来离散随机微分方程。我们应用这个框架离散和模拟ULD，它收敛到目标分布$p^{*}$。该框架不仅可用于解决对数凹抽样问题，还可用于解决涉及模拟（随机）微分方程的任何问题。

英文摘要：

Sampling from log-concave distributions is a well researched problem that has many applications in statistics and machine learning. We study the distributions of the form $p^{*}\propto\exp(-f(x))$, where $f:\mathbb{R}^{d}\rightarrow\mathbb{R}$ has an $L$-Lipschitz gradient and is $m$-strongly convex. In our paper, we propose a Markov chain Monte Carlo (MCMC) algorithm based on the underdamped Langevin diffusion (ULD). It can achieve $ε\cdot D$ error (in 2-Wasserstein distance) in $\tilde{O}\left(κ^{7/6}/ε^{1/3}+κ/ε^{2/3}\right)$ steps, where $D\overset{\mathrm{def}}{=}\sqrt{\frac{d}{m}}$ is the effective diameter of the problem and $κ\overset{\mathrm{def}}{=}\frac{L}{m}$ is the condition number. Our algorithm performs significantly faster than the previously best known algorithm for solving this problem, which requires $\tilde{O}\left(κ^{1.5}/ε\right)$ steps. Moreover, our algorithm can be easily parallelized to require only $O(κ\log\frac{1}ε)$ parallel steps. To solve the sampling problem, we propose a new framework to discretize stochastic differential equations. We apply this framework to discretize and simulate ULD, which converges to the target distribution $p^{*}$. The framework can be used to solve not only the log-concave sampling problem, but any problem that involves simulating (stochastic) differential equations.

机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。

摘要：

英文摘要：

相关论文