首页/论文/2007.01147

arXiv 2007.011472020-07-02math.STcs.CCstat.TH

Double-Loop Unadjusted Langevin Algorithm

Paul Rolland, Armin Eftekhari, Ali Kavis, Volkan Cevher

英文摘要：

A well-known first-order method for sampling from log-concave probability distributions is the Unadjusted Langevin Algorithm (ULA). This work proposes a new annealing step-size schedule for ULA, which allows to prove new convergence guarantees for sampling from a smooth log-concave distribution, which are not covered by existing state-of-the-art convergence guarantees. To establish this result, we derive a new theoretical bound that relates the Wasserstein distance to total variation distance between any two log-concave distributions that complements the reach of Talagrand T2 inequality. Moreover, applying this new step size schedule to an existing constrained sampling algorithm, we show state-of-the-art convergence rates for sampling from a constrained log-concave distribution, as well as improved dimension dependence.

机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。

阅读原文

阅读 PDF ↗

arXiv 页面 ↗HTML 原文 ↗

本文内容

英文摘要相关论文

机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。

继续探索

相关论文

Unadjusted Langevin algorithm with multiplicative noise: Total variation and Wasserstein bounds

In this paper, we focus on non-asymptotic bounds related to the Euler scheme of an ergodic diffusion with a possibly multiplicative diffusion term (non-constant diffusion coefficient). More precisely, the objective of this paper is to control the distance of the standard Euler scheme with decreasing step ({usually called Unadjusted Langevin Algorithm in the Monte Carlo literature}) to the invariant distribution of such an ergodic diffusion. In an appropriate Lyapunov setting and under {uniform} ellipticity assumptions on the diffusion coefficient, we establish (or improve) such bounds for Tota…

2022-09-22math.PRmath.STstat.TH

Heavy-tailed Sampling via Transformed Unadjusted Langevin Algorithm

We analyze the oracle complexity of sampling from polynomially decaying heavy-tailed target densities based on running the Unadjusted Langevin Algorithm on certain transformed versions of the target density. The specific class of closed-form transformation maps that we construct are shown to be diffeomorphisms, and are particularly suited for developing efficient diffusion-based samplers. We characterize the precise class of heavy-tailed densities for which polynomial-order oracle complexities (in dimension and inverse target accuracy) could be obtained, and provide illustrative examples. We h…

2022-01-20math.STstat.COstat.ML

未调整凸光滑情形下统一的HMC与欠阻尼朗之万方法

HMC and underdamped Langevin united in the unadjusted convex smooth case

该研究提出一类未调整的广义HMC采样器，统一了位置HMC与欠阻尼朗之万方法，通过部分速度刷新提升收敛速率，还建立了其随机梯度版本的无维度收敛率，拓展了相关理论结果。

2024-05-22math.PRmath.STstat.TH

Non-asymptotic convergence analysis for the Unadjusted Langevin Algorithm

In this paper, we study a method to sample from a target distribution $π$ over $\mathbb{R}^d$ having a positive density with respect to the Lebesgue measure, known up to a normalisation factor. This method is based on the Euler discretization of the overdamped Langevin stochastic differential equation associated with $π$. For both constant and decreasing step sizes in the Euler discretization, we obtain non-asymptotic bounds for the convergence to the target distribution $π$ in total variation distance. A particular attention is paid to the dependency on the dimension $d$, to demonstrate the a…

2016-12-19math.STstat.COstat.ME

Φ-散度沿无调整朗之万算法和近端采样器的快速收敛性

Fast Convergence of $Φ$-Divergence Along the Unadjusted Langevin Algorithm and Proximal Sampler

该研究将无调整朗之万算法和近端采样器的混合时间分析扩展到Φ-散度框架，证明在平稳分布满足Φ-索伯列夫不等式时，Φ-散度呈指数级快速收敛，涵盖卡方散度和相对熵混合等常见情形。

2025-02-12cs.ITmath.ITmath.ST

High-dimensional Bayesian inference via the Unadjusted Langevin Algorithm

We consider in this paper the problem of sampling a high-dimensional probability distribution $π$ having a density with respect to the Lebesgue measure on $\mathbb{R}^d$, known up to a normalization constant $x \mapsto π(x)= \mathrm{e}^{-U(x)}/\int_{\mathbb{R}^d} \mathrm{e}^{-U(y)} \mathrm{d} y$. Such problem naturally occurs for example in Bayesian inference and machine learning. Under the assumption that $U$ is continuously differentiable, $\nabla U$ is globally Lipschitz and $U$ is strongly convex, we obtain non-asymptotic bounds for the convergence to stationarity in Wasserstein distance o…

2018-07-15math.STstat.MEstat.ML

更多相关论文收起更多论文

非凸情形下改进型驯服未调整 Langevin 算法的非渐近收敛界

Non-asymptotic convergence bounds for modified tamed unadjusted Langevin algorithm in non-convex setting

针对非凸且梯度超线性增长势能的采样问题，本文提出新驯服因子的 mTULA，并证明其在 W1、W2 距离下分别以 $\mathcal{O}(\lambda)$ 和 $\mathcal{O}(\lambda^{1/2})$ 非渐近收敛。

2024-06-12cs.NAmath.NAmath.OC

随机中点朗之万蒙特卡洛方法的收敛速度

Convergence rate of randomized midpoint Langevin Monte Carlo

本文针对随机中点朗之万蒙特卡洛（RLMC），建立常步长版本的指数遍历性，设计递减步长RLMC并给出其函数类距离下的收敛速度，为该算法的理论分析提供关键结果。

2025-11-17math.PRmath.STstat.TH

Higher Order Langevin Monte Carlo Algorithm

A new (unadjusted) Langevin Monte Carlo (LMC) algorithm with improved rates in total variation and in Wasserstein distance is presented. All these are obtained in the context of sampling from a target distribution $π$ that has a density $\hatπ$ on $\mathbb{R}^d$ known up to a normalizing constant. Moreover, $-\log \hatπ$ is assumed to have a locally Lipschitz gradient and its third derivative is locally Hölder continuous with exponent $β\in (0,1]$. Non-asymptotic bounds are obtained for the convergence to stationarity of the new sampling method with convergence rate $1+ β/2$ in Wasserstein dis…

2019-09-25math.STstat.TH

用于稀有事件估计的马尔可夫链重要性采样中的仿射不变相互作用朗之万动力学

Affine invariant interacting Langevin dynamics in Markov chain importance sampling for rare event estimation

本文提出一种用于稀有事件估计的基于梯度的MCIS新方案ALDI-IS，利用ALDI对平滑零方差密度采样，并建立了非渐近误差界，揭示了平滑参数在采样误差与采样难度间的权衡。

2025-06-25math.STstat.TH

The shifted ODE method for underdamped Langevin MCMC

In this paper, we consider the underdamped Langevin diffusion (ULD) and propose a numerical approximation using its associated ordinary differential equation (ODE). When used as a Markov Chain Monte Carlo (MCMC) algorithm, we show that the ODE approximation achieves a $2$-Wasserstein error of $\varepsilon$ in $\mathcal{O}\big(d^{\frac{1}{3}}/\varepsilon^{\frac{2}{3}}\big)$ steps under the standard smoothness and strong convexity assumptions on the target distribution. This matches the complexity of the randomized midpoint method proposed by Shen and Lee [NeurIPS 2019] which was shown to be ord…

2021-01-25cs.NAmath.NAmath.PR

Sqrt(d) Dimension Dependence of Langevin Monte Carlo

Georgia Institute of Technology（佐治亚理工学院）；Shenzhen Institute of Artificial Intelligence and Robotics for Society（深圳人工智能与机器人研究院）；The Chinese University of Hong Kong, Shenzhen（香港中文大学（深圳））

This article considers the popular MCMC method of unadjusted Langevin Monte Carlo (LMC) and provides a non-asymptotic analysis of its sampling error in 2-Wasserstein distance. The proof is based on a refinement of mean-square analysis in Li et al. (2019), and this refined framework automates the analysis of a large class of sampling algorithms based on discretizations of contractive SDEs. Using this framework, we establish an $\tilde{O}(\sqrt{d}/ε)$ mixing time bound for LMC, without warm start, under the common log-smooth and log-strongly-convex conditions, plus a growth condition on the 3rd-…

2022-02-21cs.LGcs.NAmath.NA

仿射不变集合朗之万及其离散时间变体的几何遍历性

Geometric Ergodicity of Affine Invariant Ensemble Langevin and its Discrete Time Variants

Department of Mathematics, UCLA（加州大学洛杉矶分校数学系）

本文针对仿射不变集合朗之万动力学及其离散变体，通过构造新型李雅普诺夫函数解决其几何遍历性的开放问题，证明了正则化后方法的收敛性及离散化不变分布的弱收敛性。

2026-10-06cs.NAmath.NAmath.OC

无噪声采样算法的收敛性与正则化Wasserstein近似

Convergence of Noise-Free Sampling Algorithms with Regularized Wasserstein Proximals

University of California, Los Angeles（加州大学洛杉矶分校）；University of South Carolina（南卡罗来纳大学）

本文研究了正则化Wasserstein近似方法在采样强log-concave分布时的收敛性，证明其在Kullback-Leibler散度方面具有保证收敛性，并展示了其优于传统Langevin方法的性能。

2026-09-29cs.NAmath.NAmath.ST

通过 $Φ$-互信息的收缩表征沿 Langevin 动力学及算法的样本依赖性

Characterizing Dependence of Samples along the Langevin Dynamics and Algorithms via Contraction of $Φ$-Mutual Information

本文研究了连续空间采样马尔可夫链中样本间依赖性的衰减速度，通过证明沿整个轨迹成立强数据处理不等式，得出在强对数凹目标分布下样本间 $Φ$-互信息呈指数级收缩至零的结论。

2025-06-26cs.ITmath.ITmath.ST

使用 Plug & Play 先验的贝叶斯成像：当 Langevin 遇到 Tweedie

Bayesian imaging using Plug & Play priors: when Langevin meets Tweedie

University of Oxford（牛津大学）；Institut Universitaire de France（法国大学研究院）；Heriot-Watt University（赫瑞瓦特大学）；Maxwell Institute for Mathematical Sciences（麦克斯韦数学科学研究所）

本文针对贝叶斯成像中 Plug & Play (PnP) 先验方法缺乏收敛证明的问题，提出了两种可证明收敛的算法 PnP-ULA 和 PnP-SGD，并建立了详细的收敛保证，适用于图像去模糊、修复和去噪等任务。

2022-01-12cs.CVeess.IVmath.ST

kTULA：一种在超线性对数梯度下具有改进KL界的Langevin采样算法

kTULA: A Langevin sampling algorithm with improved KL bounds under super-linear log-gradients

Archimedes/Athena Research Centre & National Technical University of Athens（Archimedes/Athena研究机构 & 国家技术大学雅典）；School of Mathematics, The University of Edinburgh（数学系，爱丁堡大学）；The Alan Turing Institute, London, UK（艾伦·图灵研究所，伦敦，英国）；National Technical University of Athens（国家技术大学雅典）；Financial Technology Thrust, Society Hub, The Hong Kong University of Science and Technology (Guangzhou)（金融科技 thrust，社会枢纽，香港科技大学（广州））

针对深度学习中对数梯度超线性增长且不满足全局Lipschitz条件的采样问题，本文提出驯化Langevin算法kTULA，在KL散度下取得速率$2-\overlineε$的改进非渐近收敛界，并应用于双阱分布采样和神经网络优化。

2025-06-05cs.LGmath.PRmath.ST