Multi-agent Reinforcement Learning Accelerated MCMC on Multiscale Inversion Problem
The Chinese University of Hong Kong（香港中文大学）；Texas A&M University（得克萨斯农工大学）；North-Eastern Federal University（东北联邦大学）；University of California, Irvine（加州大学欧文分校）
Eric Chung, Yalchin Efendiev, Wing Tat Leung, Sai-Mang Pun, Zecheng Zhang
In this work, we propose a multi-agent actor-critic reinforcement learning (RL) algorithm to accelerate the multi-level Monte Carlo Markov Chain (MCMC) sampling algorithms. The policies (actors) of the agents are used to generate the proposal in the MCMC steps; and the critic, which is centralized, is in charge of estimating the long term reward. We verify our proposed algorithm by solving an inverse problem with multiple scales. There are several difficulties in the implementation of this problem by using traditional MCMC sampling. Firstly, the computation of the posterior distribution involves evaluating the forward solver, which is very time consuming for a problem with heterogeneous. We hence propose to use the multi-level algorithm. More precisely, we use the generalized multiscale finite element method (GMsFEM) as the forward solver in evaluating a posterior distribution in the multi-level rejection procedure. Secondly, it is hard to find a function which can generate samplings which are meaningful. To solve this issue, we learn an RL policy as the proposal generator. Our experiments show that the proposed method significantly improves the sampling process
We formulate gradient-based Markov chain Monte Carlo (MCMC) sampling as optimization on the space of probability measures, with Kullback-Leibler (KL) divergence as the objective functional. We show that an underdamped form of the Langevin algorithm performs accelerated gradient descent in this metric. To characterize the convergence of the algorithm, we construct a Lyapunov functional and exploit hypocoercivity of the underdamped Langevin algorithm. As an application, we show that accelerated rates can be obtained for a class of nonconvex functions with the Langevin algorithm.
Reinforcement Learning for Accelerated Aerodynamic Shape Optimisation
Software Competence Center Hagenberg (SCCH)（软件竞争力中心哈根贝格）；Institut für Strömungsmechanik und Wärmeübertragung, TU Wien（流体力学与传热研究所，维也纳技术大学）；CERBSim GmbH（CERBSim公司）
We propose a Markov chain Monte Carlo (MCMC) algorithm based on third-order Langevin dynamics for sampling from distributions with log-concave and smooth densities. The higher-order dynamics allow for more flexible discretization schemes, and we develop a specific method that combines splitting with more accurate integration. For a broad class of $d$-dimensional distributions arising from generalized linear models, we prove that the resulting third-order algorithm produces samples from a distribution that is at most $\varepsilon > 0$ in Wasserstein distance from the target distribution in $O\l…
ShanghaiTech University（上海科技大学）；University of Notre Dame（圣母大学）；Shanghai Institute of Microsystem and Information Technology, Chinese Academy of Sciences（中国科学院上海微系统与信息技术研究所）；University of Chinese Academy of Sciences（中国科学院大学）
Estimation of spatially-varying parameters for computationally expensive forward models governed by partial differential equations is addressed. A novel multiscale Bayesian inference approach is introduced based on deep probabilistic generative models. Such generative models provide a flexible representation by inferring on each scale a low-dimensional latent encoding while allowing hierarchical parameter generation from coarse- to fine-scales. Combining the multiscale generative model with Markov Chain Monte Carlo (MCMC), inference across scales is achieved enabling us to efficiently obtain p…
University of Cambridge（剑桥大学）；Google Research（谷歌研究院）
We introduce repriorisation, a data-dependent reparameterisation which transforms a Bayesian neural network (BNN) posterior to a distribution whose KL divergence to the BNN prior vanishes as layer widths grow. The repriorisation map acts directly on parameters, and its analytic simplicity complements the known neural network Gaussian process (NNGP) behaviour of wide BNNs in function space. Exploiting the repriorisation, we develop a Markov chain Monte Carlo (MCMC) posterior sampling algorithm which mixes faster the wider the BNN. This contrasts with the typically poor performance of MCMC in hi…
FFT-Accelerated Auxiliary Variable MCMC for Fermionic Lattice Models: A Determinant-Free Approach with $O(N\log N)$ Complexity
Department of Statistics and Data Science, UCLA（统计与数据科学系，加州大学洛杉矶分校）；Department of Physics, TUM School of Natural Sciences, Technical University of Munich（物理系，技术大学慕尼黑自然科学院）；Munich Center for Quantum Science and Technology（慕尼黑量子科学与技术中心）；Lambda, Inc（Lambda公司）
Accelerated training of deep learning surrogate models for surface displacement and flow, with application to MCMC-based history matching of CO2 storage operations
Markov Chain Monte Carlo (MCMC) sampling methods are widely used but often encounter either slow convergence or biased sampling when applied to multimodal high dimensional distributions. In this paper, we present a general framework of improving classical MCMC samplers by employing a global optimization method. The global optimization method first reduces a high dimensional search to an one dimensional geodesic to find a starting point close to a local mode. The search is accelerated and completed by using a local search method such as BFGS. We modify the target distribution by extracting a lo…
Accelerated Parallel Tempering via Neural Transports
University of Oxford（牛津大学）；University of Cambridge（剑桥大学）；Cornell University（康奈尔大学）；Xaira Therapeutics；National University of Singapore（新加坡国立大学）；University of British Columbia（不列颠哥伦比亚大学）
University of South Carolina（南卡罗来纳大学）；University of North Carolina Charlotte（北卡罗来纳大学夏洛特分校）
Markov Chain Monte Carlo (MCMC) methods have a drawback when working with a target distribution or likelihood function that is computationally expensive to evaluate, specially when working with big data. This paper focuses on Metropolis-Hastings (MH) algorithm for unimodal distributions. Here, an enhanced MH algorithm is proposed that requires less number of expensive function evaluations, has shorter burn-in period, and uses a better proposal distribution. The main innovations include the use of Bayesian optimization to reach the high probability region quickly, emulating the target distribut…
Georgia Institute of Technology（佐治亚理工学院）；The Chinese University of Hong Kong, Shenzhen（香港中文大学（深圳））；Shenzhen Institute of Artificial Intelligence and Robotics for Society（深圳人工智能与机器人研究院）
Nesterov's Accelerated Gradient (NAG) for optimization has better performance than its continuous time limit (noiseless kinetic Langevin) when a finite step-size is employed \citep{shi2021understanding}. This work explores the sampling counterpart of this phenonemon and proposes a diffusion process, whose discretizations can yield accelerated gradient-based MCMC methods. More precisely, we reformulate the optimizer of NAG for strongly convex functions (NAG-SC) as a Hessian-Free High-Resolution ODE, change its high-resolution coefficient to a hyperparameter, inject appropriate noise, and discre…
Nested Slice Sampling: Vectorized Nested Sampling for GPU-Accelerated Inference
Kavli Institute for Cosmology Cambridge（剑桥卡维利宇宙研究所）；Institute of Astronomy, University of Cambridge（剑桥大学天文研究所）；Cavendish Laboratory and Department of Engineering, University of Cambridge（剑桥大学卡文迪许实验室和工程系）
Accelerated Markov Chain Monte Carlo Using Adaptive Weighting Scheme
School of Computer Science and Technology, North University of China（中北大学计算机科学与技术学院）；School of Semiconductor and Physics, North University of China（中北大学半导体与物理学院）
We provide new gradient-based methods for efficiently solving a broad class of ill-conditioned optimization problems. We consider the problem of minimizing a function $f : \mathbb{R}^d \rightarrow \mathbb{R}$ which is implicitly decomposable as the sum of $m$ unknown non-interacting smooth, strongly convex functions and provide a method which solves this problem with a number of gradient evaluations that scales (up to logarithmic factors) as the product of the square-root of the condition numbers of the components. This complexity bound (which we prove is nearly optimal) can improve almost exp…
