The Metropolis Hastings algorithm and its multi-proposal extensions are aimed at the computation of the expectation $<π,f>$ of a function $f$ under a probability measure $π$ difficult to simulate. They consist in constructing by an appropriate acceptation/rejection procedure a Markov chain $(X_k,k\geq 0)$ with transition matrix $P$ such that $π$ is reversible with respect to $P$ and in estimating $<π,f>$ by the empirical mean $I_n(f)=\inv{n}\sum_{k=1}^n f(X_k)$. The waste-recycling Monte Carlo (WR) algorithm introduced by physicists is a modification of the Metropolis-Hastings algorithm, which makes use of all the proposals in the empirical mean, whereas the standard Metropolis-Hastings algorithm only uses the accepted proposals. In this paper, we extend the WR algorithm into a general control variate technique and exhibit the optimal choice of the control variate in terms of asymptotic variance. We also give an example which shows that in contradiction to the intuition of physicists, the WR algorithm can have an asymptotic variance larger than the one of the Metropolis-Hastings algorithm. However, in the particular case of the Metropolis-Hastings algorithm called Boltzmann algorithm, we prove that the WR algorithm is asymptotically better than the Metropolis-Hastings algorithm.
In Turitsyn, Chertkov, Vucelja (2011) a non-reversible Markov Chain Monte Carlo (MCMC) method on an augmented state space was introduced, here referred to as Lifted Metropolis-Hastings (LMH). A scaling limit of the magnetization process in the Curie-Weiss model is derived for LMH, as well as for Metropolis-Hastings (MH). The required jump rate in the high (supercritical) temperature regime equals $n^{1/2}$ for LMH, which should be compared to $n$ for MH. At the critical temperature the required jump rate equals $n^{3/4}$ for LMH and $n^{3/2}$ for MH, in agreement with experimental results of T…
Let $π$ be a positive continuous target density on $\mathbb{R}$. Let $P$ be the Metropolis-Hastings operator on the Lebesgue space $\mathbb{L}^2(π)$ corresponding to a proposal Markov kernel $Q$ on $\mathbb{R}$. When using the quasi-compactness method to estimate the spectral gap of $P$, a mandatory first step is to obtain an accurate bound of the essential spectral radius $r\_{ess}(P)$ of $P$. In this paper a computable bound of $r\_{ess}(P)$ is obtained under the following assumption on the proposal kernel: $Q$ has a bounded continuous density $q(x,y)$ on $\mathbb{R}^2$ satisfying the follow…
A well-known folklore result in the MCMC community is that the Metropolis-Hastings algorithm mixes quickly for any unimodal target, as long as the tails are not too heavy. Although we've heard this fact stated many times in conversation, we are not aware of any quantitative statement of this result in the literature, and we are not aware of any quick derivation from well-known results. The present paper patches this small gap in the literature, providing a generic bound based on the popular "drift-and-minorization" framework of Rosenthal (1995). Our main contribution is to study two sublevel s…
The classical Metropolis-Hastings (MH) algorithm can be extended to generate non-reversible Markov chains. This is achieved by means of a modification of the acceptance probability, using the notion of vorticity matrix. The resulting Markov chain is non-reversible. Results from the literature on asymptotic variance, large deviations theory and mixing time are mentioned, and in the case of a large deviations result, adapted, to explain how non-reversible Markov chains have favorable properties in these respects. We provide an application of NRMH in a continuous setting by developing the necessa…
We propose an adaptive independent Metropolis--Hastings algorithm with the ability to learn from all previous proposals in the chain except the current location. It is an extension of the independent Metropolis--Hastings algorithm. Convergence is proved provided a strong Doeblin condition is satisfied, which essentially requires that all the proposal functions have uniformly heavier tails than the stationary distribution. The proof also holds if proposals depending on the current state are used intermittently, provided the information from these iterations is not used for adaption. The algorit…
We study two types of Metropolis-Hastings (MH) reversiblizations for non-reversible Markov chains with Markov kernel $P$. While the first type is the classical Metropolised version of $P$, we introduce a new self-adjoint kernel which captures the opposite transition effect of the first type, that we call the second MH kernel. We investigate the spectral relationship between $P$ and the two MH kernels. Along the way, we state a version of Weyl's inequality for the spectral gap of $P$ (and hence its additive reversiblization), as well as an expansion of $P$. Both results are expressed in terms o…
We investigate local MCMC algorithms, namely the random-walk Metropolis and the Langevin algorithms, and identify the optimal choice of the local step-size as a function of the dimension $n$ of the state space, asymptotically as $n\to\infty$. We consider target distributions defined as a change of measure from a product law. Such structures arise, for instance, in inverse problems or Bayesian contexts when a product prior is combined with the likelihood. We state analytical results on the asymptotic behavior of the algorithms under general conditions on the change of measure. Our theory is mot…
Given a target distribution $μ$ and a proposal chain with generator $Q$ on a finite state space, in this paper we study two types of Metropolis-Hastings (MH) generator $M_1(Q,μ)$ and $M_2(Q,μ)$ in a continuous-time setting. While $M_1$ is the classical MH generator, we define a new generator $M_2$ that captures the opposite movement of $M_1$ and provide a comprehensive suite of comparison results ranging from hitting time and mixing time to asymptotic variance, large deviations and capacity, which demonstrate that $M_2$ enjoys superior mixing properties than $M_1$. To see that $M_1$ and $M_2$ …
The Metropolis-adjusted Langevin algorithm (MALA) is a Metropolis-Hastings method for approximate sampling from continuous distributions. We derive upper bounds for the contraction rate in Kantorovich-Rubinstein-Wasserstein distance of the MALA chain with semi-implicit Euler proposals applied to log-concave probability measures that have a density w.r.t. a Gaussian reference measure. For sufficiently "regular" densities, the estimates are dimension-independent, and they hold for sufficiently small step sizes $h$ that do not depend on the dimension either. In the limit $h\downarrow0$, the bound…
This paper defines an approximation scheme for a solution of the Poisson equation of a geometrically ergodic Metropolis-Hastings chain $Φ$. The approximations give rise to a natural sequence of control variates for the ergodic average $S_k(F)=(1/k)\sum_{i=1}^{k} F(Φ_i)$, where $F$ is the force function in the Poisson equation. The main result of the paper shows that the sequence of the asymptotic variances (in the CLTs for the control-variate estimators) converges to zero and gives a rate of this convergence. Numerical examples in the case of a double-well potential are discussed.
It has been shown that the nonreversible overdamped Langevin dynamics enjoy better convergence properties in terms of spectral gap and asymptotic variance than the reversible one. In this article we propose a variance reduction method for the Metropolis-Hastings Adjusted Langevin Algorithm (MALA) that makes use of the good behaviour of the these nonreversible dynamics. It consists in constructing a nonreversible Markov chain (with respect to the target invariant measure) by using a Generalized Metropolis-Hastings adjustment on a lifted state space. We present two variations of this method and …
A well-known difficult problem regarding Metropolis-Hastings algorithms is to get sharp bounds on their convergence rates. Moreover, a fundamental but often overlooked problem in Markov chain theory is to study the convergence rates for different initializations. In this paper, we study the two issues mentioned above of the Independent Metropolis-Hastings (IMH) algorithms on both general and discrete state spaces. We derive the exact convergence rate and prove that the IMH algorithm's different deterministic initializations have the same convergence rate. We get the exact convergence speed for…
We study a class of Metropolis-Hastings algorithms for target measures that are absolutely continuous with respect to a large class of non-Gaussian prior measures on Banach spaces. The algorithm is shown to have a spectral gap in a Wasserstein-like semimetric weighted by a Lyapunov function. A number of error bounds are given for computationally tractable approximations of the algorithm including bounds on the closeness of Cesáro averages and other pathwise quantities via perturbation theory. Several applications illustrate the breadth of problems to which the results apply such as various lik…
Motivated by Bayesian inference with highly informative data we analyze the performance of random walk-like Metropolis-Hastings algorithms for approximate sampling of increasingly concentrating target distributions. We focus on Gaussian proposals which use a Hessian-based approximation of the target covariance. By means of pushforward transition kernels we show that for Gaussian target measures the spectral gap of the corresponding Metropolis-Hastings algorithm is independent of the concentration of the posterior, i.e., the noise level in the observational data that is used for Bayesian infere…
Couplings play a central role in the analysis of Markov chain convergence and in the construction of novel Markov chain Monte Carlo estimators, diagnostics, and variance reduction techniques. The set of possible couplings is often intractable, frustrating the search for tight bounds and efficient estimators. To address this challenge for algorithms in the Metropolis-Hastings (MH) family, we establish a simple characterization of the set of MH transition kernel couplings. We then extend this result to describe the set of maximal couplings of the MH kernel, resolving an open question of O'Leary …
For sufficiently smooth targets of product form it is known that the variance of a single coordinate of the proposal in RWM (Random walk Metropolis) and MALA (Metropolis adjusted Langevin algorithm) should optimally scale as $n^{-1}$ and as $n^{-\frac{1}{3}}$ with dimension $n$, and that the acceptance rates should be tuned to $0.234$ and $0.574$. We establish counterexamples to demonstrate that smoothness assumptions of the order of $\mathcal{C}^1(\mathbb{R})$ for RWM and $\mathcal{C}^3(\mathbb{R})$ for MALA are indeed required if these scaling rates are to hold. The counterexamples identify …
We consider a pseudo-marginal Metropolis--Hastings kernel $P_m$ that is constructed using an average of $m$ exchangeable random variables, as well as an analogous kernel $P_s$ that averages $s<m$ of these same random variables. Using an embedding technique to facilitate comparisons, we show that the asymptotic variances of ergodic averages associated with $P_m$ are lower bounded in terms of those associated with $P_s$. We show that the bound provided is tight and disprove a conjecture that when the random variables to be averaged are independent, the asymptotic variance under $P_m$ is never le…
This work develops a powerful and versatile framework for determining acceptance ratios in Metropolis-Hastings type Markov kernels widely used in statistical sampling problems. Our approach allows us to derive new classes of kernels which unify random walk or diffusion-type sampling methods with more complicated "extended phase space" algorithms based around ideas from Hamiltonian dynamics. Our starting point is an abstract result developed in the generality of measurable state spaces that addresses proposal kernels that possess a certain involution structure. Note that, while this underlying …
Given a target distribution $μ$ on a general state space $\mathcal{X}$ and a proposal Markov jump process with generator $Q$, the purpose of this paper is to investigate two universal properties enjoyed by two types of Metropolis-Hastings (MH) processes with generators $M_1(Q,μ)$ and $M_2(Q,μ)$ respectively. First, we motivate our study of $M_2$ by offering a geometric interpretation of $M_1$, $M_2$ and their convex combinations as $L^1$ minimizers between $Q$ and the set of $μ$-reversible generators of Markov jump processes. Second, specializing into the case of $\mathcal{X} = \mathbb{R}^d$ a…
We analyse computational efficiency of Metropolis-Hastings algorithms with AR(1) process proposals. These proposals include, as a subclass, discretized Langevin diffusion (e.g. MALA) and discretized Hamiltonian dynamics (e.g. HMC). By including the effect of Metropolis-Hastings we extend earlier work by Fox and Parker, who used matrix splitting techniques to analyse the performance and improve efficiency of AR(1) processes for targeting Gaussian distributions. Our research enables analysis of MCMC methods that draw samples from non-Gaussian target distributions by using AR(1) process proposals…
A classical approach for approximating expectations of functions w.r.t. partially known distributions is to compute the average of function values along a trajectory of a Metropolis-Hastings (MH) Markov chain. A key part in the MH algorithm is a suitable acceptance/rejection of a proposed state, which ensures the correct stationary distribution of the resulting Markov chain. However, the rejection of proposals causes highly correlated samples. In particular, when a state is rejected it is not taken any further into account. In contrast to that we consider a MH importance sampling estimator whi…
In this paper we study the ergodicity properties of some adaptive Markov chain Monte Carlo algorithms (MCMC) that have been recently proposed in the literature. We prove that under a set of verifiable conditions, ergodic averages calculated from the output of a so-called adaptive MCMC sampler converge to the required value and can even, under more stringent assumptions, satisfy a central limit theorem. We prove that the conditions required are satisfied for the independent Metropolis--Hastings algorithm and the random walk Metropolis algorithm with symmetric increments. Finally, we propose an …
We consider the random walk Metropolis algorithm on $\mathbb{R}^n$ with Gaussian proposals, and when the target probability measure is the $n$-fold product of a one-dimensional law. In the limit $n\to\infty$, it is well known (see [Ann. Appl. Probab. 7 (1997) 110-120]) that, when the variance of the proposal scales inversely proportional to the dimension $n$ whereas time is accelerated by the factor $n$, a diffusive limit is obtained for each component of the Markov chain if this chain starts at equilibrium. This paper extends this result when the initial distribution is not the target probabi…
We study the problem of sampling high and infinite dimensional target measures arising in applications such as conditioned diffusions and inverse problems. We focus on those that arise from approximating measures on Hilbert spaces defined via a density with respect to a Gaussian reference measure. We consider the Metropolis-Hastings algorithm that adds an accept-reject mechanism to a Markov chain proposal in order to make the chain reversible with respect to the target measure. We focus on cases where the proposal is either a Gaussian random walk (RWM) with covariance equal to that of the refe…
The choice of the increment distribution is crucial for the random-walk Metropolis-Hastings (RWM) algorithm. In this paper we study the optimal choice in high-dimension setting among all possible increment distributions. The conclusion is rather counter intuitive, but the optimal rate of convergence is attained by the usual choice, the normal distribution as the increment distribution. In particular, no heavy-tailed increment distribution can improve the rate.
Proposals for Metropolis-Hastings MCMC derived by discretizing Langevin diffusion or Hamiltonian dynamics are examples of stochastic autoregressive proposals that form a natural wider class of proposals with equivalent computability. We analyze Metropolis-Hastings MCMC with stochastic autoregressive proposals applied to target distributions that are absolutely continuous with respect to some Gaussian distribution to derive expressions for expected acceptance probability and expected jump size, as well as measures of computational cost, in the limit of high dimension. Thus, we are able to unify…
We study a Monte Carlo algorithm for simulation of probability distributions based on stochastic step functions, and compare to the traditional Metropolis/Hastings method. Unlike the latter, the step function algorithm can produce an uncorrelated Markov chain. We apply this method to the simulation of Levy processes, for which simulation of uncorrelated jumps are essential. We perform numerical tests consisting of simulation from probability distributions, as well as simulation of Levy process paths. The Levy processes include a jump-diffusion with a Gaussian Levy measure, as well as jump-diff…
The Monte Carlo within Metropolis (MCwM) algorithm, interpreted as a perturbed Metropolis-Hastings (MH) algorithm, provides an approach for approximate sampling when the target distribution is intractable. Assuming the unperturbed Markov chain is geometrically ergodic, we show explicit estimates of the difference between the n-th step distributions of the perturbed MCwM and the unperturbed MH chains. These bounds are based on novel perturbation results for Markov chains which are of interest beyond the MCwM setting. To apply the bounds, we need to control the difference between the transition …
Equilibrium systems evolve according to Detailed Balance (DB). This principe guided development of the Monte-Carlo sampling techniques, of which Metropolis-Hastings (MH) algorithm is the famous representative. It is also known that DB is sufficient but not necessary. We construct irreversible deformation of a given reversible algorithm capable of dramatic improvement of sampling from known distribution. Our transformation modifies transition rates keeping the structure of transitions intact. To illustrate the general scheme we design an Irreversible version of Metropolis-Hastings (IMH) and tes…
