Let X={X_n:n=0,1,2,...} be an irreducible, positive recurrent Markov chain with invariant probability measure π. We show that if X satisfies a one-step minorization condition, then πcan be represented as an infinite mixture. The distributions in the mixture are associated with the hitting times on an accessible atom introduced via the splitting construction of Athreya and Ney [Trans. Amer. Math. Soc. 245 (1978) 493-501] and Nummelin [Z. Wahrsch. Verw. Gebiete 43 (1978) 309-318]. When the small set in the minorization condition is the entire state space, our mixture representation of πreduces to a simple formula, first derived by Breyer and Roberts [Methodol. Comput. Appl. Probab. 3 (2001) 161-177] from which samples can be easily drawn. Despite the fact that the derivation of this formula involves no coupling or backward simulation arguments, the formula can be used to reconstruct perfect sampling algorithms based on coupling from the past
This paper concerns the Vertex Reinforced Jump Process (VRJP) and its representations as a Markov process in random environment. We show that all possible representations of the VRJP as a mixture of Markov processes can be expressed in a similar form, using a random potential and harmonic functions for an associated operator. This allows to show that the VRJP on $\mathbb{Z}^d$ (with certain initial conditions) has a unique representation, by proving that an associated Martin boundary is trivial. Moreover, on infinite trees, we construct a family of representations, that are all different when …
New results on conditional joint probability distributions of first exit times are presented for a continuous-time stochastic process defined as the mixture of Markov jump processes moving at different speeds on the same finite state space, while the mixture occurs at a random time. Such mixture was first proposed by Frydman \cite{Frydman2005} and Frydman and Schuermann \cite{Frydman2008} as a generalization of the mover-stayer model of Blumen et at. \cite{Blumen}, and was recently extended by Surya \cite{Surya2018}, in which explicit distributional identities of the process are given, in part…
This paper presents some new results on the conditional joint probability distributions of phase-type under the mixture of right-continuous Markov jump processes with absorption on the same finite state space $\mathbb{S}$ moving at different speeds, where the mixture occurs at a random time. Such mixture was first proposed by Frydman \cite{Frydman2005} and Frydman and Schuermann \cite{Frydman2008} as a generalization of the mover-stayer model of Blumen et at. \cite{Blumen}, and was recently extended by Surya \cite{Surya2018}. When conditioning on all previous and current information $\mathcal{…
This paper generalizes the work of Kendall [Electron. Comm. Probab. 9 (2004) 140--151], which showed that perfect simulation, in the form of dominated coupling from the past, is always possible (although not necessarily practical) for geometrically ergodic Markov chains. Here, we consider the more general situation of positive recurrent chains and explore when it is possible to produce such a simulation algorithm for these chains. We introduce a class of chains which we name tame, for which we show that perfect simulation is possible.
Partially exchangeable sequences representable as mixtures of Markov chains are completely specified by de Finetti's mixing measure. The paper characterizes, in terms of a subclass of hidden Markov models, the partially exchangeable sequences with mixing measure concentrated on a countable set, both for discrete valued and for Polish valued sequences.
We show that if the density of the absolutely continuous part of a copula is bounded away from zero on a set of Lebesgue measure 1, then that copula generates \textquotedblleft lower $ψ$-mixing\textquotedblright\ stationary Markov chains. This conclusion implies $ϕ$-mixing, $ρ$-mixing, $β$-mixing and \textquotedblleft interlaced $ρ$-mixing\textquotedblright . We also provide some new results on the mixing structure of Markov chains generated by mixtures of copulas.
Predictive constructions are a powerful way of characterizing the probability law of stochastic processes with certain forms of invariance, such as exchangeability or Markov exchangeability. When de Finetti-like representation theorems are available, the predictive characterization implicitly defines the prior distribution, starting from assumptions on the observables; moreover, it often helps designing efficient computational strategies. In this paper we give necessary and sufficient conditions on the sequence of predictive distributions such that they characterize a Markov exchangeable proba…
For many probability distributions of interest, it is quite difficult to obtain samples efficiently. Often, Markov chains are employed to obtain approximately random samples from these distributions. The primary drawback to traditional Markov chain methods is that the mixing time of the chain is usually unknown, which makes it impossible to determine how close the output samples are to having the target distribution. Here we present a new protocol, the randomness recycler (RR), that overcomes this difficulty. Unlike classical Markov chain approaches, an RR-based algorithm creates samples drawn…
Cutoff for mixtures of permuted Markov chains: reversible case
Département d’Informatique de l’ENS, École Normale Supérieure（巴黎高等师范学院计算机科学系）；CNRS（法国国家科学研究中心）；PSL Research University（巴黎文理研究大学）；INRIA Paris（法国国家信息与自动化研究所巴黎分部）；CNRS, I2M, Aix-Marseille Université（法国国家科学研究中心、马赛数学研究所、艾克斯-马赛大学）
We show that combinations of optimal (stationary) policies in unichain Markov decision processes are optimal. That is, let M be a unichain Markov decision process with state space S, action space A and policies π_j^*: S -> A (1\leq j\leq n) with optimal average infinite horizon reward. Then any combination πof these policies, where for each state i in S there is a j such that π(i)=π_j^*(i), is optimal as well. Furthermore, we prove that any mixture of optimal policies, where at each visit in a state i an arbitrary action π_j^*(i) of an optimal policy is chosen, yields optimal average reward, t…
Perfect sampling is a technique that uses coupling arguments to provide a sample from the stationary distribution of a Markov chain in a finite time without ever computing the distribution. This technique is very efficient if all the events in the system have monotonicity property. However, in the general (non-monotone) case, this technique needs to consider the whole state space, which limits its application only to chains with a state space of small cardinality. We propose here a new approach for the general case that only needs to consider two trajectories. Instead of the original chain, we…
Bounding chains are a technique that offers three benefits to Markov chain practitioners: a theoretical bound on the mixing time of the chain under restricted conditions, experimental bounds on the mixing time of the chain that are provably accurate and construction of perfect sampling algorithms when used in conjunction with protocols such as coupling from the past. Perfect sampling algorithms generate variates exactly from the target distribution without the need to know the mixing time of a Markov chain at all. We present here the basic theory and use of bounding chains for several chains f…
In this paper we consider two-point unilateral Markov fields on a two-dimensional lattice as considered by Pickard, Galbraith and Walley. We show that, under various ergodicity conditions, they can be perfectly simulated in the stationary state on any finite window. The techniques which are used connect perfect simulation with oriented percolation through suitable coupling constructions.
We provide the first perfect sampling algorithm for a Generalized Jackson Network of FIFO queues under arbitrary topology and non-Markovian assumptions on the input of the network. We assume (in addition to stability) that the interarrival and service times of customers have finite moment generating function in a neighborhood of the origin, and the interarrival times have unbounded support.
We present a perfect sampling algorithm for Gibbs point processes, based on the partial rejection sampling of Guo et al. (2017). Our particular focus is on pairwise interaction processes, penetrable spheres mixture models and area-interaction processes, with a finite interaction range. For an interaction range $2r$ of the target process, the proposed algorithm can generate a perfect sample with $O(\log(1/r))$ expected running time complexity, provided that the intensity of the points is not too high.
We present the first algorithm for generating random variates exactly uniformly from the set of perfect matchings of a bipartite graph with a polynomial expected running time over a nontrivial set of graphs. Previous Markov chain approaches obtain approximately uniform variates for arbitrary graphs in polynomial time, but their general running time is $Θ(n^{26} (\ln n)^2).$ Our algorithm employs acceptance/rejection together with a new upper limit on the permanent of a form similar to Bregman's Theorem. For a graph with $2n$ nodes where the degree of every node is nearly $γn$ for a constant $γ…
We consider a particle system on $Z^d$ with finite state space and interactions of infinite range. Assuming that the rate of change is continuous and decays sufficiently fast, we introduce a perfect simulation algorithm for the stationary process. The algorithm follows from a representation of the multicolor system as a finitary coding from a sequence of independent uniform random variables. This implies that the process is exponentially ergodic. The basic tool we use is a representation of the infinite range change rates as a mixture of finite range change rates.
In this article we describe a new coupling technique which is useful in a variety of perfect sampling algorithms. A multishift coupler generates a random function f() so that for each real x, f(x)-x is governed by the same fixed probability distribution, such as a normal distribution. We develop the class of layered multishift couplers, which are simple and have several useful properties. For the standard normal distribution, for instance, the layered multishift coupler generates an f() which (surprisingly) maps an interval of length L to fewer than 2+L/2.35 points --- useful in applications w…
We describe a new algorithm for the perfect simulation of variable length Markov chains and random systems with perfect connections. This algorithm, which generalizes Propp and Wilson's simulation scheme, is based on the idea of coupling into and from the past. It improves on existing algorithms by relaxing the conditions on the kernel and by accelerating convergence, even in the simple case of finite order Markov chains. Although chains of variable or infinite order have been widely investigated for decades, their use in applied probability, from information theory to bio-informatics and ling…
Given a $p$-coin that lands heads with unknown probability $p$, we wish to produce an $f(p)$-coin for a given function $f: (0,1) \rightarrow (0,1)$. This problem is commonly known as the Bernoulli Factory and results on its solvability and complexity have already been obtained. Nevertheless, generic ways to design a practical Bernoulli Factory for a given function $f$ exist only in a few special cases. We present a constructive way to build an efficient Bernoulli Factory when $f(p)$ is a rational function with coefficients in $\mathbb{R}$. Moreover, we extend the Bernoulli Factory problem to a…
A probabilistic cellular automaton (PCA) can be viewed as a Markov chain. The cells are updated synchronously and independently, according to a distribution depending on a finite neighborhood. We investigate the ergodicity of this Markov chain. A classical cellular automaton is a particular case of PCA. For a 1-dimensional cellular automaton, we prove that ergodicity is equivalent to nilpotency, and is therefore undecidable. We then propose an efficient perfect sampling algorithm for the invariant measure of an ergodic PCA. Our algorithm does not assume any monotonicity property of the local r…
This paper analyzes the performance of sequential importance sampling algorithms for estimating the number of perfect matchings in bipartite graphs. Precise bounds on the number of samples required to yield an accurate estimate are derived. In doing so, moments of permutation statistics are computed using generating functions and nonstandard limit theorems are derived by expressing perfect matchings as a time-inhomogeneous Markov chain.
We obtain discrete mixture representations for parametric families of probability distributions on Euclidean spheres, such as the von Mises--Fisher, the Watson and the angular Gaussian families. In addition to several special results we present a general approach to isotropic distribution families that is based on density expansions in terms of special surface harmonics. We discuss the connections to stochastic processes on spheres, in particular random walks, discrete mixture representations derived from spherical diffusions, and the use of Markov representations for the mixing base to obtain…
