We study the convergence to equilibrium of an underdamped Langevin equation that is controlled by a linear feedback force. Specifically, we are interested in sampling the possibly multimodal invariant probability distribution of a Langevin system at small noise (or low temperature), for which the dynamics can easily get trapped inside metastable subsets of the phase space. We follow [Chen et al., J. Math. Phys. 56, 113302, 2015] and consider a Langevin equation that is simulated at a high temperature, with the control playing the role of a friction that balances the additional noise so as to restore the original invariant measure at a lower temperature. We discuss different limits as the temperature ratio goes to infinity and prove convergence to a limit dynamics. It turns out that, depending on whether the lower ("target") or the higher ("simulation") temperature is fixed, the controlled dynamics converges either to the overdamped Langevin equation or to a deterministic gradient flow. This implies that (a) the ergodic limit and the large temperature separation limit do not commute in general, and that (b) it is not possible to accelerate the speed of convergence to the ergodic limit by making the temperature separation larger and larger. We discuss the implications of these observation from the perspective of stochastic optimisation algorithms and enhanced sampling schemes in molecular dynamics.
Gradient is All You Need? How Consensus-Based Optimization can be Interpreted as a Stochastic Relaxation of Gradient Descent
University of Oxford, Mathematical Institute（牛津大学数学研究所）；Deeptech Consulting（德普科技咨询）；ETH Zurich, Department of Mathematics（苏黎世联邦理工学院数学系）；Technical University of Munich, School of Computation, Information and Technology, Department of Mathematics（慕尼黑技术大学计算、信息与技术学院数学系）；Munich Center for Machine Learning Munich Data Science Institute（慕尼黑机器学习中心慕尼黑数据科学研究所）
Convex Relaxation for Solving Large-Margin Classifiers in Hyperbolic Space
John A. Paulson School of Engineering and Applied Sciences, Harvard University（哈佛大学约翰·A·保尔森工程与应用科学学院）；Center for Brain Science, Harvard University（哈佛大学脑科学中心）；Kempner Institute for the Study of Natural and Artificial Intelligence, Harvard University（哈佛大学肯普ner自然与人工智能研究所）
Global Convergence of Stochastic Gradient Descent for Some Non-convex Matrix Problems
Stanford University（斯坦福大学）
低秩分解上的随机梯度下降（SGD）通常用于加速矩阵问题，包括矩阵完成，子空间跟踪和SDP松弛。在本文中，我们展示了一个步长计划SGD的低秩最小二乘问题，我们证明了，在广泛的采样条件下，我们的方法收敛全局从一个随机的起始点内$O(ε^{-1} n \log n)$步骤与常数概率的常秩问题。我们的SGD修改涉及到随机幂迭代。我们还展示了实验来说明该算法的运行时间和收敛性。
