用于约束采样的惩罚型过阻尼与欠阻尼 Langevin Monte Carlo 算法
Penalized Overdamped and Underdamped Langevin Monte Carlo Algorithms for Constrained Sampling
AI总结：
本文提出基于惩罚函数的 PLD/PULMC 及随机梯度变体，将凸体约束采样转为无约束采样，并给出 TV 与 W2 距离下的非凸和强凸收敛保证。
摘要：
我们考虑约束采样问题，其目标是在 x 被约束位于凸体 C 内时，从目标分布 π(x)∝e^{-f(x)} 中采样。受连续优化中惩罚方法的启发，我们提出惩罚型 Langevin Dynamics（PLD）和惩罚型欠阻尼 Langevin Monte Carlo（PULMC）方法，通过为约束违反引入惩罚函数，将约束采样问题转化为无约束采样问题。当 f 光滑且梯度可用时，我们得到 PLD 以 ε 误差采样目标分布的迭代复杂度为 \ ilde{O}(d/ε^{10})，其中误差以 TV 距离衡量，\ ilde{O}(·) 隐藏对数因子。对于 PULMC，当 f 的 Hessian 具有 Lipschitz 性且 C 的边界足够光滑时，我们将结果改进为 \ ilde{O}(√d/ε^7)。据我们所知，这是欠阻尼 Langevin Monte Carlo 方法在约束采样中的首批收敛结果，可处理非凸 f，并在现有使用确定性梯度的方法中提供具有最佳维度依赖的保证。如果可获得 f 梯度的无偏随机估计，我们提出 PSGLD 和 PSGULMC 方法，它们能够处理随机梯度并可扩展到大型数据集，而无需 Metropolis-Hasting 校正步骤。对于 PSGLD 和 PSGULMC，当 f 强凸且光滑时，我们在 W2 距离下分别获得 \ ilde{O}(d/ε^{18}) 和 \ ilde{O}(d√d/ε^{39}) 的迭代复杂度。当 f 光滑且可以非凸时，我们提供有限时间性能界和迭代复杂度结果。最后，我们在 Bayesian LASSO 回归和 Bayesian 约束深度学习问题上展示其性能。
英文摘要：
We consider the constrained sampling problem where the goal is to sample from a target distribution $π(x)\propto e^{-f(x)}$ when $x$ is constrained to lie on a convex body $\mathcal{C}$. Motivated by penalty methods from continuous optimization, we propose penalized Langevin Dynamics (PLD) and penalized underdamped Langevin Monte Carlo (PULMC) methods that convert the constrained sampling problem into an unconstrained sampling problem by introducing a penalty function for constraint violations. When $f$ is smooth and gradients are available, we get $\tilde{\mathcal{O}}(d/\varepsilon^{10})$ iteration complexity for PLD to sample the target up to an $\varepsilon$-error where the error is measured in the TV distance and $\tilde{\mathcal{O}}(\cdot)$ hides logarithmic factors. For PULMC, we improve the result to $\tilde{\mathcal{O}}(\sqrt{d}/\varepsilon^{7})$ when the Hessian of $f$ is Lipschitz and the boundary of $\mathcal{C}$ is sufficiently smooth. To our knowledge, these are the first convergence results for underdamped Langevin Monte Carlo methods in the constrained sampling that handle non-convex $f$ and provide guarantees with the best dimension dependency among existing methods with deterministic gradient. If unbiased stochastic estimates of the gradient of $f$ are available, we propose PSGLD and PSGULMC methods that can handle stochastic gradients and are scaleable to large datasets without requiring Metropolis-Hasting correction steps. For PSGLD and PSGULMC, when $f$ is strongly convex and smooth, we obtain $\tilde{\mathcal{O}}(d/\varepsilon^{18})$ and $\tilde{\mathcal{O}}(d\sqrt{d}/\varepsilon^{39})$ iteration complexity in W2 distance. When $f$ is smooth and can be non-convex, we provide finite-time performance bounds and iteration complexity results. Finally, we illustrate the performance on Bayesian LASSO regression and Bayesian constrained deep learning problems.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
