关于用于模拟退火的广义Langevin方程

Convergence of Langevin-Simulated Annealing algorithms with multiplicative noise

We study the convergence of Langevin-Simulated Annealing type algorithms with multiplicative noise, i.e. for $V : \mathbb{R}^d \to \mathbb{R}$ a potential function to minimize, we consider the stochastic equation $dY_t = - σσ^\top \nabla V(Y_t) dt + a(t)σ(Y_t)dW_t + a(t)^2Υ(Y_t)dt$, where $(W_t)$ is a Brownian motion, where $σ: \mathbb{R}^d \to \mathcal{M}_d(\mathbb{R})$ is an adaptive (multiplicative) noise, where $a : \mathbb{R}^+ \to \mathbb{R}^+$ is a function decreasing to $0$ and where $Υ$ is a correction term. This setting can be applied to optimization problems arising in Machine Learn…

On the convergence of an improved and adaptive kinetic simulated annealing

Inspired by the work of [Fang et al.. An improved annealing method and its large-time behaviour. Stochastic Process. Appl. (1997), Volume 71 Issue 1 Page 55-74.], who propose an improved simulated annealing algorithm based on a variant of overdamped Langevin diffusion with state-dependent diffusion coefficient, we cast this idea in the kinetic setting and develop an improved kinetic simulated annealing (IKSA) method for minimizing a target function $U$. To analyze its convergence, we utilize the framework recently introduced by [Monmarché. Hypocoercivity in metastable settings and kinetic simu…

Convergence of the kinetic annealing for general potentials

The convergence of the kinetic Langevin simulated annealing is proven under mild assumptions on the potential $U$ for slow logarithmic cooling schedules. Moreover, non-convergence for fast logarithmic and non-logarithmic cooling schedules is established. The results are based on an adaptation to non-elliptic non-reversible kinetic settings of a localization/local convergence strategy developed by Fournier and Tardif in the overdamped elliptic case, and on precise quantitative high order Sobolev hypocoercive estimates.

退火欠阻尼朗之万动力学

Annealed Underdamped Langevin Dynamics

本文提出退火欠阻尼朗之万采样方法，针对对数凹分布，通过辅助过程克服布朗运动退化问题，获得总变差距离下 $\mathcal{O}(\epsilon^{-2})$ 的迭代复杂度，优于退火过阻尼方法的 $\mathcal{O}(\epsilon^{-6})$，并有望推广至非对数凹目标。

受控群体梯度动力学

Controlled Swarm Gradient Dynamics

本文提出受控群体梯度动力学方法，通过受控冷却计划实现全局优化，利用不变测度作为退火曲线，确保快速收敛到全局极小点。

Uniform Generalization Bound on Time and Inverse Temperature for Gradient Descent Algorithm and its Application to Analysis of Simulated Annealing

In this paper, we propose a novel uniform generalization bound on the time and inverse temperature for stochastic gradient Langevin dynamics (SGLD) in a non-convex setting. While previous works derive their generalization bounds by uniform stability, we use Rademacher complexity to make our generalization bound independent of the time and inverse temperature. Using Rademacher complexity, we can reduce the problem to derive a generalization bound on the whole space to that on a bounded region and therefore can remove the effect of the time and inverse temperature from our generalization bound. …

时间非齐次随机微分方程的Fisher信息耗散

Fisher information dissipation for time inhomogeneous stochastic differential equations

该研究针对时间非齐次变系数随机微分方程开展Lyapunov收敛分析，通过修正梯度流表述、选取时间依赖相对Fisher信息泛函并提出Hessian矩阵条件，证明了三类典型Langevin动力学的收敛性并通过数值算例验证。

无维度多模态采样：基于预处理退火朗之万动力学

Dimension-Free Multimodal Sampling via Preconditioned Annealed Langevin Dynamics

针对函数空间问题有限维近似细化下保持稳定的多模态目标采样算法，提出预处理退火朗之万动力学，在谱条件下实现维度一致的时间复杂度内的高斯混合目标采样。

离散最优传输：退火算法的快速收敛性

Discrete Optimal Transport: Rapid Convergence of Simulated Annealing Algorithms

本文提出离散最优传输框架，分析有限状态空间上的退火算法收敛性，通过定义离散Wasserstein-2距离和离散作用，证明退火算法的KL散度可由离散作用控制，并应用于Ising模型和Potts模型，获得多项式上界。

Ergodicity of the infinite swapping algorithm at low temperature

Sampling Gibbs measures at low temperatures is an important task but computationally challenging. Numerical evidence suggests that the infinite-swapping algorithm (isa) is a promising method. The isa can be seen as an improvement of the replica methods. We rigorously analyze the ergodic properties of the isa in the low temperature regime, deducing an Eyring-Kramers formula for the spectral gap (or Poincaré constant) and an estimate for the log-Sobolev constant. Our main results indicate that the effective energy barrier can be reduced drastically using the isa compared to the classical overdam…

隐偏约化与提升

Hypocoercivity meets lifts

本文将隐偏约化框架与二阶提升结合，展示其在动力学方程中的应用，并证明自适应 Langevin 动力学在近二次势能下的近最优性。

预处理退火 Langevin 动力学在多模高斯混合中的维度均匀离散化分析

Dimension-Uniform Discretization Analysis of Preconditioned Annealed Langevin Dynamics for Multimodal Gaussian Mixtures

本文研究了预处理退火 Langevin 动力学在高斯混合中的稳定性问题，通过 Euler-Maruyama 离散化和指数积分方案，证明了在满足特定谱条件时，KL 散度具有维度均匀的上界。