Convergence of Unadjusted Langevin in High Dimensions: Delocalization of Bias
Courant Institute, New York University, NY, USA（纽约大学Courant研究所）；Department of Mathematics, University of California, Los Angeles, CA, USA（加州大学洛杉矶分校数学系）
Yifan Chen, Xiaoou Cheng, Jonathan Niles-Weed, Jonathan Weare
The unadjusted Langevin algorithm is commonly used to sample probability distributions in extremely high-dimensional settings. However, existing analyses of the algorithm for strongly log-concave distributions suggest that, as the dimension $d$ of the problem increases, the number of iterations required to ensure convergence within a desired error in the $W_2$ metric scales in proportion to $d$ or $\sqrt{d}$. In this paper, we argue that, despite this poor scaling of the $W_2$ error for the full set of variables, the behavior for a small number of variables can be significantly better: a number of iterations proportional to $K$, up to logarithmic terms in $d$, often suffices for the algorithm to converge to within a desired $W_2$ error for all $K$-marginals. We refer to this effect as delocalization of bias. We show that the delocalization effect does not hold universally and prove its validity for Gaussian distributions and strongly log-concave distributions with certain sparse interactions. Our analysis relies on a novel $W_{2,\ell^\infty}$ metric to measure convergence. A key technical challenge we address is the lack of a one-step contraction property in this metric. Finally, we use asymptotic arguments to explore potential generalizations of the delocalization effect beyond the Gaussian and sparse interactions setting.
Delocalization of bias in unadjusted Hamiltonian Monte Carlo and underdamped Langevin
Department of Mathematics, University of California, Los Angeles, CA 90095, USA（加州大学洛杉矶分校数学系）；Courant Institute, New York University, NY 10012, USA（纽约大学Courant研究所）
Rutgers Business School（罗杰斯商学院）；Department of Mathematics（数学系）；School of Mathematics and Statistics（数学与统计学学院）；Department of Management Science and Information Systems（管理科学与信息系统系）
Taming neural networks with TUSLA: Non-convex learning via adaptive stochastic gradient Langevin algorithms
Alfréd Rényi Institute of Mathematics（阿尔弗雷德·雷尼数学研究所）；The University of Edinburgh（爱丁堡大学）；The Alan Turing Institute（艾伦·图灵研究所）；National Technical University of Athens（雅典国立技术大学）
Error estimates for tamed Euler and Randomized Euler schemes for SDEs with locally Lipschitz drift with applications to non-logconcave sampling and optimization
Archimedes, Athena Research Center（阿基米德雅典研究中心）；National Technical University of Athens（雅典国立技术大学）
Athena/Archimedes Research Centre（雅典娜/阿基米德研究中心）；The University of Edinburgh（爱丁堡大学）；Univ. Grenoble Alpes（格勒诺布尔大学）；CNRS（法国国家科学研究中心）；Inria（法国国家信息与自动化研究所）；Grenoble INP（格勒诺布尔工业大学）；LIG（格勒诺布尔信息与信号处理实验室）
Sqrt(d) Dimension Dependence of Langevin Monte Carlo
Georgia Institute of Technology（佐治亚理工学院）；Shenzhen Institute of Artificial Intelligence and Robotics for Society（深圳人工智能与机器人研究院）；The Chinese University of Hong Kong, Shenzhen（香港中文大学（深圳））
本文考虑了流行的MCMC方法的未调整的朗之万蒙特卡罗（LMC），并提供了一个非渐近分析的抽样误差在2-Wasserstein距离。该证明基于Li et al.（2019）中的均方分析的改进，并且这种改进的框架自动分析了基于收缩性SDEs离散化的一大类采样算法。在此框架下，我们建立了LMC的$\tilde{O}(\sqrt{d}/ε)$混合时间界，在无热启动的情况下，在常用的对数光滑和对数强凸条件下，加上目标测度势的三阶导数的增长条件.该界限改进了先前已知的最佳$\tilde{O}(d/ε)$结果，并且对于满足上述假设的目标测量，在维度$d$和精度容限$ε$方面都是最优的（就阶而言）。数值实验进一步验证了我们的理论分析。
Department of Statistics, University of Oxford（1 统计系，牛津大学）；Inria, CNRS, École Normale Supérieure, PSL Research University（2 Inria，法国国家科学研究中心，巴黎高等师范学院，PSL研究大学）；Department of Mathematical Sciences, University of Copenhagen（3 数学科学系，哥本哈根大学）
NF-ULA: Langevin Monte Carlo with Normalizing Flow Prior for Imaging Inverse Problems
Shanghai Jiao Tong University（上海交通大学）；University of Birmingham（伯明翰大学）；Indian Institute of Technology (IIT) Kharagpur（印度理工学院卡拉格布尔分校）；University of Cambridge（剑桥大学）
Bayesian imaging using Plug & Play priors: when Langevin meets Tweedie
University of Oxford（牛津大学）；Institut Universitaire de France（法国大学研究院）；Heriot-Watt University（赫瑞瓦特大学）；Maxwell Institute for Mathematical Sciences（麦克斯韦数学科学研究所）
本文针对贝叶斯成像中 Plug & Play (PnP) 先验方法缺乏收敛证明的问题，提出了两种可证明收敛的算法 PnP-ULA 和 PnP-SGD，并建立了详细的收敛保证，适用于图像去模糊、修复和去噪等任务。
