Markov chain Monte Carlo is a method of producing a correlated sample in order to estimate features of a target distribution via ergodic averages. A fundamental question is when should sampling stop? That is, when are the ergodic averages good estimates of the desired quantities? We consider a method that stops the simulation when the width of a confidence interval based on an ergodic average is less than a user-specified value. Hence calculating a Monte Carlo standard error is a critical step in assessing the simulation output. We consider the regenerative simulation and batch means methods of estimating the variance of the asymptotic normal distribution. We give sufficient conditions for the strong consistency of both methods and investigate their finite sample properties in a variety of examples.
We elaborate the idea behind Markov chain Monte Carlo (MCMC) methods in a mathematically coherent, yet simple and understandable way. To this end, we proof a pivotal convergence theorem for finite Markov chains and a minimal version of the Perron-Frobenius theorem. Subsequently, we briefly discuss two fundamental MCMC methods, the Gibbs and Metropolis-Hastings sampler. Only very basic knowledge about matrices, convergence of real sequences and probability theory is required.
Markov chain models are used in various fields, such behavioral sciences or econometrics. Although the goodness of fit of the model is usually assessed by large sample approximation, it is desirable to use conditional tests if the sample size is not large. We study Markov bases for performing conditional tests of the toric homogeneous Markov chain model, which is the envelope exponential family for the usual homogeneous Markov chain model. We give a complete description of a Markov basis for the following cases: i) two-state, arbitrary length, ii) arbitrary finite state space and length of thr…
We propose a sequential Markov chain Monte Carlo (SMCMC) algorithm to sample from a sequence of probability distributions, corresponding to posterior distributions at different times in on-line applications. SMCMC proceeds as in usual MCMC but with the stationary distribution updated appropriately each time new data arrive. SMCMC has advantages over sequential Monte Carlo (SMC) in avoiding particle degeneracy issues. We provide theoretical guarantees for the marginal convergence of SMCMC under various settings, including parametric and nonparametric models. The proposed approach is compared to…
Sequential Monte Carlo (SMC) is a methodology for sampling approximately from a sequence of probability distributions of increasing dimension and estimating their normalizing constants. We propose here an alternative methodology named Sequentially Interacting Markov Chain Monte Carlo (SIMCMC). SIMCMC methods work by generating interacting non-Markovian sequences which behave asymptotically like independent Metropolis-Hastings (MH) Markov chains with the desired limiting distributions. Contrary to SMC, SIMCMC allows us to iteratively improve our estimates in an MCMC-like fashion. We establish c…
We consider conditional exact tests of factor effects in designed experiments for discrete response variables. Similarly to the analysis of contingency tables, a Markov chain Monte Carlo method can be used for performing exact tests, when large-sample approximations are poor and the enumeration of the conditional sample space is infeasible. For designed experiments with a single observation for each run, we formulate log-linear or logistic models and consider a connected Markov chain over an appropriate sample space. In particular, we investigate fractional factorial designs with $2^{p-q}$ run…
We present several Monte Carlo strategies for simulating discrete-time Markov chains with continuous multi-dimensional state space; we focus on stratified techniques. We first analyze the variance of the calculation of the measure of a domain included in the unit hypercube, when stratified samples are used. We then show that each step of the simulation of a Markov chain can be reduced to the numerical integration of the indicator function of a subdomain of the unit hypercube. Our approach for Markov chains simulates N copies of the chain in parallel using stratified sampling and the copies are…
The random numbers driving Markov chain Monte Carlo (MCMC) simulation are usually modeled as independent U(0,1) random variables. Tribble [Markov chain Monte Carlo algorithms using completely uniformly distributed driving sequences (2007) Stanford Univ.] reports substantial improvements when those random numbers are replaced by carefully balanced inputs from completely uniformly distributed sequences. The previous theoretical justification for using anything other than i.i.d. U(0,1) points shows consistency for estimated means, but only applies for discrete stationary distributions. We extend …
We consider quantile estimation using Markov chain Monte Carlo and establish conditions under which the sampling distribution of the Monte Carlo error is approximately Normal. Further, we investigate techniques to estimate the associated asymptotic variance, which enables construction of an asymptotically valid interval estimator. Finally, we explore the finite sample properties of these methods through examples and provide some recommendations to practitioners.
This paper presents a novel theoretical Monte Carlo Markov chain procedure in the framework of graphs. It specifically deals with the construction of a Markov chain whose empirical distribution converges to a given reference one. The Markov chain is constrained over an underlying graph, so that states are viewed as vertices and the transition between two states can have positive probability only in presence of an edge connecting them. The analysis is carried out on the basis of the relationship between the support of the target distribution and the connectedness of the graph.
This paper proposes a family of weighted batch means variance estimators, which are computationally efficient and can be conveniently applied in practice. The focus is on Markov chain Monte Carlo simulations and estimation of the asymptotic covariance matrix in the Markov chain central limit theorem, where conditions ensuring strong consistency are provided. Finite sample performance is evaluated through auto-regressive, Bayesian spatial-temporal, and Bayesian logistic regression examples, where the new estimators show significant computational gains with a minor sacrifice in variance compared…
Markov chain Monte Calro methods (MCMC) are commonly used in Bayesian statistics. In the last twenty years, many results have been established for the calculation of the exact convergence rate of MCMC methods. We introduce another rate of convergence for MCMC methods by approximation techniques. This rate can be obtained by the convergence of the Markov chain to a diffusion process. We apply it to a simple mixture model and obtain its convergence rate. Numerical simulations are performed to illustrate the effect of the rate.
Historically time-reversibility of the transitions or processes underpinning Markov chain Monte Carlo methods (MCMC) has played a key rôle in their development, while the self-adjointness of associated operators together with the use of classical functional analysis techniques on Hilbert spaces have led to powerful and practically successful tools to characterize and compare their performance. Similar results for algorithms relying on nonreversible Markov processes are scarce. We show that for a type of nonreversible Monte Carlo Markov chains and processes, of current or renewed interest in th…
Current reporting of results based on Markov chain Monte Carlo computations could be improved. In particular, a measure of the accuracy of the resulting estimates is rarely reported. Thus we have little ability to objectively assess the quality of the reported estimates. We address this issue in that we discuss why Monte Carlo standard errors are important, how they can be easily calculated in Markov chain Monte Carlo and how they can be used to decide when to stop the simulation. We compare their use to a popular alternative in the context of two examples.
We show fundamental properties of the Markov semigroup of recently proposed MCMC algorithms based on Piecewise-deterministic Markov processes (PDMPs) such as the Bouncy Particle Sampler, the Zig-Zag process or the Randomized Hamiltonian Monte Carlo method. Under assumptions typically satisfied in MCMC settings, we prove that PDMPs are Feller and that their generator admits the space of infinitely differentiable functions with compact support as a core. As we illustrate via martingale problems and a simplified proof of the invariance of target distributions, these results provide a fundamental …
We consider Markov chain Monte Carlo methods for calculating conditional p values of statistical models for count data arising in Box-Behnken designs. The statistical model we consider is a discrete version of the first-order model in the response surface methodology. For our models, the Markov basis, a key notion to construct a connected Markov chain on a given sample space, is characterized as generators of the toric ideals for the centrally symmetric configurations of root system D_n. We show the structure of the Groebner bases for these cases. A numerical example for an imaginary data set …
We describe ergodic properties of some Metropolis-Hastings (MH) algorithms for heavy-tailed target distributions. The analysis usually falls into sub-geometric ergodicity framework but we prove that the mixed preconditioned Crank-Nicolson (MpCN) algorithm has geometric ergodicity even for heavy-tailed target distributions. This useful property comes from the fact that the MpCN algorithm becomes a random-walk Metropolis algorithm under suitable transformation.
An introduction to the use of linchpin variables in Markov chain Monte Carlo (MCMC) is provided. Before the widespread adoption of MCMC methods, conditional sampling using linchpin variables was essentially the only practical approach for simulating from multivariate distributions. With the advent of MCMC, linchpin variables were largely ignored. However, there has been a resurgence of interest in using them in conjunction with MCMC methods and there are good reasons for doing so. A simple derivation of the method is provided, its validity, benefits, and limitations are discussed, and some exa…
This paper offers a personal review of some things we've learned about rates of convergence of Markov chains to their stationary distributions. The main topic is ways of speeding up diffusive behavior. It also points to open problems and how much more there is to do.
Markov chain Monte Carlo (MCMC) simulations are commonly employed for estimating features of a target distribution, particularly for Bayesian inference. A fundamental challenge is determining when these simulations should stop. We consider a sequential stopping rule that terminates the simulation when the width of a confidence interval is sufficiently small relative to the size of the target parameter. Specifically, we propose relative magnitude and relative standard deviation stopping rules in the context of MCMC. In each setting, we develop sufficient conditions for asymptotic validity, that…
It is known that a Markov basis of the binary graph model of a graph $G$ corresponds to a set of binomial generators of cut ideals $I_{\widehat{G}}$ of the suspension $\widehat{G}$ of $G$. In this paper, we give another application of cut ideals to statistics. We show that a set of binomial generators of cut ideals is a Markov basis of some regular two-level fractional factorial design. As application, we give a Markov basis of degree 2 for designs defined by at most two relations.
Markov chain Monte Carlo (MCMC) produces a correlated sample for estimating expectations with respect to a target distribution. A fundamental question is when should sampling stop so that we have good estimates of the desired quantities? The key to answering this question lies in assessing the Monte Carlo error through a multivariate Markov chain central limit theorem (CLT). The multivariate nature of this Monte Carlo error largely has been ignored in the MCMC literature. We present a multivariate framework for terminating simulation in MCMC. We define a multivariate effective sample size, est…
We study asymptotic behavior of Monte Carlo method. Local consistency is one of an ideal property of Monte Carlo method. However, it may fail to hold local consistency for several reason. In fact, in practice, it is more important to study such a non-ideal behavior. We call local degeneracy for one of a non-ideal behavior of Monte Carlo methods. We show some equivalent conditions for local degeneracy. As an application we study a Gibbs sampler (data augmentation) for cumulative logit model with or without marginal augmentation. It is well known that natural Gibbs sampler does not work well for…
In this paper, we introduce the notion of efficiency (consistency) and examine some asymptotic properties of Markov chain Monte Carlo methods. We apply these results to the data augmentation (DA) procedure for independent and identically distributed observations. More precisely, we show that if both the sample size and the running time of the DA procedure tend to infinity the empirical distribution of the DA procedure tends to the posterior distribution. This is a local property of the DA procedure, which may be, in some cases, more helpful than the global properties to describe its behavior. …
From basic considerations of the Lie group that preserves a target probability measure, we derive the Barker, Metropolis, and ensemble Markov chain Monte Carlo (MCMC) algorithms, as well as variants of waste-recycling Metropolis-Hastings and an altogether new MCMC algorithm. We illustrate these constructions with explicit numerical computations, and we empirically demonstrate on a spin glass that the new algorithm converges more quickly than its siblings.
We study the asymptotic behavior of kernel estimators of asymptotic variances (or long-run variances) for a class of adaptive Markov chains. The convergence is studied both in $L^p$ and almost surely. The results also apply to Markov chains and improve on the existing literature by imposing weaker conditions. We illustrate the results with applications to the $\operatorname {GARCH}(1,1)$ Markov model and to an adaptive MCMC algorithm for Bayesian logistic regression.
We present a multivariate central limit theorem for a general class of interacting Markov chain Monte Carlo algorithms used to solve nonlinear measure-valued equations. These algorithms generate stochastic processes which belong to the class of nonlinear Markov chains interacting with their empirical occupation measures. We develop an original theoretical analysis based on resolvent operators and semigroup techniques to analyze the fluctuations of their occupation measures around their limiting values.
This file is the Sweave documentation for the examples provided in Flegal, J. M. and Jones, G. L. (2010), "Implementing Markov chain Monte Carlo: Estimating with confidence", in Handbook of Markov Chain Monte Carlo, edited by Brooks, S., Gelman, A., Jones, G., and Meng, X. published by Chapman & Hall/CRC Press.
Recently there have been exciting developments in Monte Carlo methods, with the development of new MCMC and sequential Monte Carlo (SMC) algorithms which are based on continuous-time, rather than discrete-time, Markov processes. This has led to some fundamentally new Monte Carlo algorithms which can be used to sample from, say, a posterior distribution. Interestingly, continuous-time algorithms seem particularly well suited to Bayesian analysis in big-data settings as they need only access a small sub-set of data points at each iteration, and yet are still guaranteed to target the true posteri…
From its inception in the 1950s to the modern frontiers of applied statistics, Markov chain Monte Carlo has been one of the most ubiquitous and successful methods in statistical computing. In that time its development has been fueled by increasingly difficult problems and novel techniques from physics. In this article I will review the history of Markov chain Monte Carlo from its inception with the Metropolis method to today's state-of-the-art in Hamiltonian Monte Carlo. Along the way I will focus on the evolving interplay between the statistical and physical perspectives of the method.
In the thesis we take the split chain approach to analyzing Markov chains and use it to establish fixed-width results for estimators obtained via Markov chain Monte Carlo procedures (MCMC). Theoretical results include necessary and sufficient conditions in terms of regeneration for central limit theorems for ergodic Markov chains and a regenerative proof of a CLT version for uniformly ergodic Markov chains with $E_πf^2< \infty.$ To obtain asymptotic confidence intervals for MCMC estimators, strongly consistent estimators of the asymptotic variance are essential. We relax assumptions required t…
Recent work incorporating geometric ideas in Markov chain Monte Carlo is reviewed in order to highlight these advances and their possible application in a range of domains beyond Statistics. A full exposition of Markov chains and their use in Monte Carlo simulation for Statistical inference and molecular dynamics is provided, with particular emphasis on methods based on Langevin diffusions. After this geometric concepts in Markov chain Monte Carlo are introduced. A full derivation of the Langevin diffusion on a Riemannian manifold is given, together with a discussion of appropriate Riemannian …
We introduce a novel time-homogeneous Markov embedding of a class of time inhomogeneous Markov chains widely used in the context of Monte Carlo sampling algorithms which allows us to answer one of the most basic, yet hard, question about the practical implementation of these techniques. We also show that this embedding sheds some light on the recent result of [#maire-douc-olsson2013]. We discuss further applications of the technique.
We introduce interacting particle Markov chain Monte Carlo (iPMCMC), a PMCMC method based on an interacting pool of standard and conditional sequential Monte Carlo samplers. Like related methods, iPMCMC is a Markov chain Monte Carlo sampler on an extended space. We present empirical results that show significant improvements in mixing rates relative to both non-interacting PMCMC samplers, and a single PMCMC sampler with an equivalent memory and computational budget. An additional advantage of the iPMCMC method is that it is suitable for distributed and multi-core architectures.
Markov chain Monte Carlo (MCMC) is a simulation method commonly used for estimating expectations with respect to a given distribution. We consider estimating the covariance matrix of the asymptotic multivariate normal distribution of a vector of sample means. Geyer (1992) developed a Monte Carlo error estimation method for estimating a univariate mean. We propose a novel multivariate version of Geyer's method that provides an asymptotically valid estimator for the covariance matrix and results in stable Monte Carlo estimates. The finite sample properties of the proposed method are investigated…
The aim of this work is to give an introduction to the theoretical background and computational complexity of Markov chain Monte Carlo methods. Most of the mathematical results related to the convergence are not found in most of the statistical references, and computational complexity is still an open question for most of the MCMC methods. In this work, we provide a general overview, references, and discussion about all these theoretical subjects.
In most sampling algorithms, including Hamiltonian Monte Carlo, transition rates between states correspond to the probability of making a transition in a single time step, and are constrained to be less than or equal to 1. We derive a Hamiltonian Monte Carlo algorithm using a continuous time Markov jump process, and are thus able to escape this constraint. Transition rates in a Markov jump process need only be non-negative. We demonstrate that the new algorithm leads to improved mixing for several example problems, both by evaluating the spectral gap of the Markov operator, and by computing au…
We propose a method to efficiently integrate truncated probability densities. The method uses Markov chain Monte Carlo method to sample from a probability density matching the function being integrated. The required normalisation or equivalently the result is obtained by constructing a function with known integral, through non-parametric kernel density estimation and variational procedure. The method is demonstrated with numerical case studies. Possible enhancements to the method and limitations are discussed.
This is a contribution for the discussion on "Unbiased Markov chain Monte Carlo with couplings" by Pierre E. Jacob, John O'Leary and Yves F. Atchadé to appear in the Journal of the Royal Statistical Society Series B.
Markov chain Monte Carlo is an inherently serial algorithm. Although likelihood calculations for individual steps can sometimes be parallelized, the serial evolution of the process is widely viewed as incompatible with parallelization, offering no speedup for samplers which require large numbers of iterations to converge to equilibrium. We provide a methodology for parallelizing Markov chain Monte Carlo across large numbers of independent, asynchronous processors. Our approach uses a partitioning and weight estimation scheme to combine independent simulations run on separate processors into ri…
This is the compilation of our comments submitted to the Journal of the Royal Statistical Society, Series B, to be published within the discussion of the Read Paper of Andrieu, Doucet and Hollenstein.
We introduce an efficient nonreversible Markov chain Monte Carlo algorithm to generate self-avoiding walks with a variable endpoint. In two dimensions, the new algorithm slightly outperforms the two-move nonreversible Berretti-Sokal algorithm introduced by H.~Hu, X.~Chen, and Y.~Deng in \cite{old}, while for three-dimensional walks, it is 3--5 times faster. The new algorithm introduces nonreversible Markov chains that obey global balance and allows for three types of elementary moves on the existing self-avoiding walk: shorten, extend or alter conformation without changing the walk's length.
Phylogenetic inference is an intractable statistical problem on a complex space. Markov chain Monte Carlo methods are the primary tool for Bayesian phylogenetic inference but it is challenging to construct efficient schemes to explore the associated posterior distribution or assess their performance. Existing approaches are unable to diagnose mixing or convergence of Markov schemes jointly across all components of a phylogenetic model. Lagged couplings of Markov chain Monte Carlo algorithms have recently been developed on simpler spaces to diagnose convergence and construct unbiased estimators…
Approximate Bayesian computation allows for inference of complicated probabilistic models with intractable likelihoods using model simulations. The Markov chain Monte Carlo implementation of approximate Bayesian computation is often sensitive to the tolerance parameter: low tolerance leads to poor mixing and large tolerance entails excess bias. We consider an approach using a relatively large tolerance for the Markov chain Monte Carlo sampler to ensure its sufficient mixing, and post-processing the output leading to estimators for a range of finer tolerances. We introduce an approximate confid…
Markov chain Monte Carlo is a class of algorithms for drawing Markovian samples from high-dimensional target densities to approximate the numerical integration associated with computing statistical expectation, especially in Bayesian statistics. However, many Markov chain Monte Carlo algorithms do not seem to share the same theoretical support and each algorithm is proven in a different way. This incurs many terminologies and ancillary concepts, which makes Markov chain Monte Carlo literature seems to be scattered and intimidating to researchers from many other fields, including new researcher…
