Small-world MCMC and convergence to multi-modal distributions: From slow mixing to fast mixing

非对数凹分布下MCMC算法的快速条件混合

Fast Conditional Mixing of MCMC Algorithms for Non-log-concave Distributions

本文针对非对数凹分布下MCMC算法全局混合缓慢的问题，提出基于庞加莱型不等式在状态空间子集上成立的条件混合理论，证明MCMC迭代的条件分布可快速混合到真实条件分布，并展示了其在高斯混合采样、参数估计和Gibbs采样中的应用价值。

How to Combine Fast Heuristic Markov Chain Monte Carlo with Slow Exact Sampling

Use each of n exact samples as the initial state for a MCMC sampler run for m steps. We give confidence intervals for accuracy of estimators which are always valid and which, in certain settings, are almost as good as the intervals one would obtain if the (unknown) mixing time of the chain were known.

Ergodicity of Approximate MCMC Chains with Applications to Large Data Sets

In many modern applications, difficulty in evaluating the posterior density makes performing even a single MCMC step slow. This difficulty can be caused by intractable likelihood functions, but also appears for routine problems with large data sets. Many researchers have responded by running approximate versions of MCMC algorithms. In this note, we develop quantitative bounds for showing the ergodicity of these approximate samplers. We then use these bounds to study the bias-variance trade-off of approximate MCMC algorithms. We apply our results to simple versions of recently proposed algorith…

Parallel and interacting Markov chains Monte Carlo method

In many situations it is important to be able to propose $N$ independent realizations of a given distribution law. We propose a strategy for making $N$ parallel Monte Carlo Markov Chains (MCMC) interact in order to get an approximation of an independent $N$-sample of a given target law. In this method each individual chain proposes candidates for all other chains. We prove that the set of interacting chains is itself a MCMC method for the product of $N$ target measures. Compared to independent parallel chains this method is more time consuming, but we show through concrete examples that it pos…

Sampling for Bayesian Mixture Models: MCMC with Polynomial-Time Mixing

We study the problem of sampling from the power posterior distribution in Bayesian Gaussian mixture models, a robust version of the classical posterior. This power posterior is known to be non-log-concave and multi-modal, which leads to exponential mixing times for some standard MCMC algorithms. We introduce and study the Reflected Metropolis-Hastings Random Walk (RMRW) algorithm for sampling. For symmetric two-component Gaussian mixtures, we prove that its mixing time is bounded as $d^{1.5}(d + \Vert θ_{0} \Vert^2)^{4.5}$ as long as the sample size $n$ is of the order $d (d + \Vert θ_{0} \Ver…

Exponentially slow mixing in the mean-field Swendsen-Wang dynamics

Swendsen-Wang dynamics for the Potts model was proposed in the late 1980's as an alternative to single-site heat-bath dynamics, in which global updates allow this MCMC sampler to switch between metastable states and ideally mix faster. Gore and Jerrum (1999) found that this dynamics may in fact exhibit slow mixing: they showed that, for the Potts model with $q\geq 3$ colors on the complete graph on $n$ vertices at the critical point $β_c(q)$, Swendsen-Wang dynamics has $t_{\mathrm{mix}}\geq \exp(c\sqrt n)$. The same lower bound was extended to the critical window $(β_s,β_S)$ around $β_c$ by Ga…