超越对数凹性的ULMC方法Wasserstein收敛的统一框架：旧与新

中文摘要 AI 辅助

作为计算统计、科学计算和机器学习中的一项基本任务，从高维概率分布中采样近年来受到越来越多的关注。人们提出了许多采样算法，其中基于欠阻尼Langevin动力学（ULD）的欠阻尼Langevin蒙特卡洛（ULMC）方法已成为一类高效算法。在这项工作中，我们引入了一个“通用”的预测-校正公式，通过不同的方法参数选择，将Euler型、UBU型和随机化方案联系起来。值得注意的是，该“通用”积分器催生了两类新型低成本积分器，称为低成本随机化积分器（LC-RIs）和低成本UBU积分器（LC-UBUIs），以及基于多项式和有理逼近的无指数变体。由此产生的新型UBU型和随机化方案每次迭代仅需一次梯度评估和两个高斯样本，大大减少了现有对应方案每次迭代所需的梯度评估或高斯样本数量。此外，我们为概率度量下的一般离散化方案建立了一个长时间误差分析的通用框架。在一定的光滑性和非对数凹性条件下，我们依靠该统一框架建立了新旧方案的非渐近$\mathcal{W}_1$误差界，揭示了Euler型方案的收敛阶为$\mathcal{O}(d^{\frac{1}{2}}h)$，UBU型方案的收敛阶为$\mathcal{O}(d h^2)$，随机化方案的收敛阶为$\mathcal{O}(d^{\frac{1}{2}}h^{\frac{3}{2}})$。在强凸设置下，可以在$\mathcal{W}_2$距离下恢复相同的非渐近误差界。数值实验证实了理论发现。

英文摘要

As a fundamental task across computational statistics, scientific computing and machine learning, sampling from high-dimensional probability distributions has received increasing attention in recent years. Numerous sampling algorithms have been proposed, among which underdamped Langevin Monte Carlo (ULMC) methods based on underdamped Langevin dynamics (ULD) have emerged as a class of efficient ones. In this work, we introduce a ``universal" predictor-corrector formulation that bridges Euler-type, UBU-type and randomized schemes through different choices of method parameters. Notably, the ``universal" integrator induces two novel classes of low-cost integrators, termed low-cost randomized integrators (LC-RIs) and low-cost UBU integrators (LC-UBUIs), as well as their exponential-free variants based on polynomial and rational approximations. The resulting new UBU-type and randomized schemes require only one gradient evaluation and two Gaussians per iteration, considerably reducing the number of gradient evaluations or Gaussians per iteration required by existing counterparts. Further, a general framework of long-time error analysis is developed for general discretization schemes in a probability metric. Under certain smoothness and non-log-concavity conditions, we rely on the unified framework to establish non-asymptotic $\mathcal{W}_1$-error bounds of both old and new schemes, revealing convergence rates of order $\mathcal{O}(d^{\frac{1}{2}}h)$ for Euler-type schemes, order $\mathcal{O}(d h^2)$ for UBU-type ones and order $\mathcal{O}(d^{\frac{1}{2}}h^{\frac{3}{2}})$ for randomized ones. In the strongly convex setting, the same non-asymptotic error bounds can be recovered in $\mathcal{W}_2$-distance. Numerical experiments corroborate the theoretical findings.

发表机构

• Central South University（中南大学）

机构由 AI 辅助整理，请以论文原文为准。