AI 中文总结
针对带噪声的零阶采样场景，提出LMC-SPSA方法，优化了Wasserstein误差的维度依赖，精度缩放更优，实证采样误差更小。
AI 中文摘要
在采样问题中，基于梯度的方法如朗之万蒙特卡洛（LMC）的混合速度快于非梯度方法，但其适用性受限于能否获取目标对数密度的梯度。实际场景中梯度往往不可用，函数评估还带有噪声，例如随机模拟器或黑箱模拟器。因此，我们提出带噪声的LMC-SPSA方法，该方法每次迭代通过两次带噪声的函数评估来近似目标对数密度的梯度。我们证明，在梯度估计存在噪声的情况下，通过证明Wasserstein距离的收敛性，带噪声的LMC-SPSA在分布上收敛；进一步构造了递减步长策略，该策略仍能将Wasserstein误差界驱动至收敛，将收敛保证扩展至恒定步长之外的场景。此外，我们将Wasserstein误差的主导维度依赖从$O(p^4)$优化为$O(p^2)$（其中$p$表示维度），并通过数值结果支持该分析。我们表明，带噪声的LMC-SPSA实现$W_2$精度$\boldsymbol{\text{ε}}$所需的总噪声-神谕复杂度为$O(p/\boldsymbol{\text{ε}}^2+\boldsymbol{\text{δ}}^2p^3/\boldsymbol{\text{ε}}^3)$，其中$\boldsymbol{\text{δ}}$是配对噪声水平，这相较于Roy等人提出的ZO-LMC方法，改善了依赖噪声的精度缩放关系。我们还证明，在递减步长和扰动序列下，当迭代次数趋于无穷时，Wasserstein误差渐近消失，并在带噪声的零阶反馈下推导了平衡策略的显式收敛速率。我们开展了实证实验以验证带噪声的LMC-SPSA的性能，与ZO-LMC方法进行神谕预算匹配的对比，结果显示在相同的函数评估预算下，其经验采样误差更小。
英文摘要
In sampling problems, gradient-based schemes such as Langevin Monte Carlo (LMC) mix faster than non-gradient-based methods, but their applicability is limited by access to the gradient of the target log-density. In practice, gradients are often unavailable and function evaluations are noisy, e.g., stochastic simulators or black-box simulators, so we propose LMC-SPSA with noise, which approximates the gradient of the target log-density using two noisy function evaluations per iteration. We prove, under noisy gradient estimates, that LMC-SPSA converges in distribution by proving the convergence in Wasserstein distance. Furthermore, we construct a diminishing step-size schedule that still drives the Wasserstein error bound to convergence, extending convergence guarantees beyond the constant-step setting. Further, we sharpen the dominant dimension dependence of the Wasserstein error from $O(p^4)$ to $O(p^2)$ (with $p$ denoting the dimension), and support this analysis with numerical results. We show that LMC-SPSA achieves $W_2$-accuracy $\varepsilon$ with total noisy-oracle complexity of $O(p/\varepsilon^2+δ^2p^3/\varepsilon^3)$, where $δ$ is the paired-noise level. This improves the noise-dependent accuracy scaling relative to the ZO-LMC method of Roy et al. We further establish asymptotically vanishing Wasserstein error as the number of iterations $\to\infty$ under diminishing step-size and perturbation sequences and derive an explicit convergence rate for a balanced schedule under noisy zeroth-order feedback. Empirical experiments are conducted to verify the performance of LMC-SPSA with noise. We provide an oracle-budget-matched comparison with the ZO-LMC method, showing smaller empirical sampling errors under the same function-evaluation budget.
Comments33 pages, 4 figures
