Many finite-state reversible Markov chains can be naturally decomposed into "projection" and "restriction" chains. In this paper we provide bounds on the total variation mixing times of the original chain in terms of the mixing properties of these related chains. This paper is in the tradition of existing bounds on Poincare and log-Sobolev constants of Markov chains in terms of similar decompositions [JSTV04, MR02, MR06, MY09]. Our proofs are simple, relying largely on recent results relating hitting and mixing times of reversible Markov chains [PS13, Oli12]. We describe situations in which our results give substantially better bounds than those obtained by applying existing decomposition results and provide examples for illustration.
We consider finite-state Markov chains that can be naturally decomposed into smaller ``projection'' and ``restriction'' chains. Possibly this decomposition will be inductive, in that the restriction chains will be smaller copies of the initial chain. We provide expressions for Poincare (resp. log-Sobolev) constants of the initial Markov chain in terms of Poincare (resp. log-Sobolev) constants of the projection and restriction chains, together with further a parameter. In the case of the Poincare constant, our bound is always at least as good as existing ones and, depending on the value of the …
Mean-field integro-differential equations are studied in an abstract framework, through couplings of the corresponding stochastic processes. In the perturbative regime, the equation is proven to admit a unique equilibrium, toward which the process converges exponentially fast. Similarly, in this case, the associated particle system is proven to converge toward its equilibrium at a rate independent from the number of particles.
In an influential paper, Courtois and Semal (1984) establish that when $G$ is an irreducible substochastic matrix for which $\sum_{n=0}^{\infty}G^n <\infty$, then the stationary distribution of any stochastic matrix $P\ge G$ can be expressed as a convex combination of the normalized rows of $(I-G)^{-1} = \sum_{n=0}^{\infty} G^n$. In this note, we give a short proof of this result that extends the theory to the countably infinite and continuous state space settings. This result plays an important role in obtaining error bounds in algorithms involving nearly decomposable Markov chains, and also …
The problem of controlling a finite state Markov chain in the presence of an adversary so as to ensure desired performance levels for a vector of objectives is cast in the framework of Blackwell approachability. Relying on an elementary two time scale construction a control scheme is proposed which ensures almost sure convergence to the desired set regardless of the adversarial actions.
Large deviation theory is a branch of probability theory that is devoted to a study of the "rate" at which empirical estimates of various quantities converge to their true values. The object of study in this paper is the rate at which estimates of the doublet frequencies of a Markov chain over a finite alphabet converge to their true values. In case the Markov process is actually an i.i.d.\ process, the rate function turns out to be the relative entropy (or Kullback-Leibler divergence) between the true and the estimated probability vectors. This result is a special case of a very general resul…
We observe that the technique of Markov contraction can be used to establish measure concentration for a broad class of non-contracting chains. In particular, geometric ergodicity provides a simple and versatile framework. This leads to a short, elementary proof of a general concentration inequality for Markov and hidden Markov chains (HMM), which supercedes some of the known results and easily extends to other processes such as Markov trees. As applications, we give a Dvoretzky-Kiefer-Wolfowitz-type inequality and a uniform Chernoff bound. All of our bounds are dimension-free and hold for cou…
It is shown that irreducible two-state continuous-time Markov chains interacting on a network in a bilinear fashion have a unique stable steady state. The proof is elementary and uses the relative entropy function.
Bifurcating Markov chains (BMC) are Markov chains indexed by a full binary tree representing the evolution of a trait along a population where each individual has two children. We provide a central limit theorem for additive functionals of BMC under $L^2$-ergodic conditions with three different regimes. This completes the pointwise approach developed in a previous work. As application, we study the elementary case of symmetric bifurcating autoregressive process, which justify the non-trivial hypothesis considered on the kernel transition of the BMC. We illustrate in this example the phase tran…
Squaring and adding $\pm 1$ mod p generates a curiously intractable random walk. A similar process over the finite field $\mathbf{F}_q$ (with $q=2^d$) leads to novel connections between elementary Galois theory and probability.
Let $X_n$ be a discrete time Markov chain with state space $S$ (countably infinite, in general) and initial probability distribution $μ^{(0)} = (P(X_0=i_1),P(X_0=i_2),\cdots,)$. What is the probability of choosing in random some $k \in \mathbb{N}$ with $k \leq n$ such that $X_k = j$ where $j \in S$? This probability is the average $\frac{1}{n} \sum_{k=1}^n μ^{(k)}_j$ where $μ^{(k)}_j = P(X_k = j)$. In this note we will study the limit of this average without assuming that the chain is irreducible, using elementary mathematical tools. Finally, we study the limit of the average $\frac{1}{n} \sum…
We develop randomized modifications of Markov chains and apply these modifications to the routing chains of customers in Jacksonian stochastic networks. The aim of our investigations is to find new rerouting schemes for non standard Jackson networks which hitherto resist computing explicitly the stationary distribution. The non standard properties we can handle by suitable algorithms encompass several modifications of Jackson networks known in the literature, especially breakdown and repair of nodes with access modification for customers to down nodes, finite buffers with control of buffer ove…
This paper presents an elementary proof of stochastic stability of a discrete-time reversible Markov chain starting from a Foster-Lyapunov drift condition. Besides its relative simplicity, there are two salient features of the proof: (i) it relies entirely on functional-analytic non-probabilistic arguments; and (ii) it makes explicit the connection between a Foster-Lyapunov function and Poincaré inequality. The proof is used to derive an explicit bound for the spectral gap. An extension to the non-reversible case is also presented.
We give an elementary construction of a time-invertible Markov process which is discrete except at one instance. The process is one of the quadratic harnesses studied in our previous papers and can be regarded as a random joint of two independent Poisson processes.
The Central Limit Theorem (CLT) for additive functionals of Markov chains is a well known result with a long history. In this paper we present applications to two finite-memory versions of the Elephant Random Walk, solving a problem from arXiv:1812.01915. We also present a derivation of the CLT for additive functionals of finite state Markov chains, which is based on positive recurrence, the CLT for IID sequences and some elementary linear algebra, and which focuses on characterization of the variance.
We give a short and elementary proof of a $(q, μ, ν)$-deformed Binomial distribution identity arising in the study of the $(q, μ, ν)$-Boson process and the $(q, μ, ν)$-TASEP. This identity found by Corwin in [4] was a key technical step to prove an intertwining relation between the Markov transition matrices of these two classes of discrete-time Markov chains. This was used in turn to derive exact formulas for a large class of observables of both these processes.
Using elementary methods, we prove that for a countable Markov chain $P$ of ergodic degree $d > 0$ the rate of convergence towards the stationary distribution is subgeometric of order $n^{-d}$, provided the initial distribution satisfies certain conditions of asymptotic decay. An example, modelling a renewal process and providing a markovian approximation scheme in dynamical system theory, is worked out in detail, illustrating the relationships between convergence behaviour, analytic properties of the generating functions associated to transition probabilities and spectral properties of the Ma…
We propose the study of Markov chains on groups as a "quasi-isometry invariant" theory that encompasses random walks. In particular, we focus on certain classes of groups acting on hyperbolic spaces including (non-elementary) hyperbolic and relatively hyperbolic groups, acylindrically hyperbolic 3-manifold groups, as well as fundamental groups of certain graphs of groups with edge groups of subexponential growth. For those, we prove a linear progress result and various applications, and these lead to a Central Limit Theorem for random walks on groups quasi-isometric to the ones we consider.
Motivated by queues with many servers, we study Brownian steady-state approximations for continuous time Markov chains (CTMCs). Our approximations are based on diffusion models (rather than a diffusion limit) whose steady-state, we prove, approximates that of the Markov chain with notable precision. Strong approximations provide such "limitless" approximations for process dynamics. Our focus here is on steady-state distributions, and the diffusion model that we propose is tractable relative to strong approximations. Within an asymptotic framework, in which a scale parameter $n$ is taken large,…
Consider a discrete finite-dimensional, Markovian market model. In this setting, discretely sampled American options can be priced using the so-called ``non-recombining'' tree algorithm. By successively increasing the number of exercise times, the American option price itself can be computed; for combinatorial reasons, we shall consider a recursive algorithm that doubles the number of exercise times at each recursion step. First we prove, by elementary arguments, error bounds for the first order differences in this recursive algorithm. From this, bounds on the higher order differences can be o…
We consider the Fleming--Viot particle system associated with a continuous-time Markov chain in a finite space. Assuming irreducibility, it is known that the particle system possesses a unique stationary distribution, under which its empirical measure converges to the quasistationary distribution of the Markov chain. We complement this Law of Large Numbers with a Central Limit Theorem. Our proof essentially relies on elementary computations on the infinitesimal generator of the Fleming--Viot particle system, and involves the so-called $π$-return process in the expression of the asymptotic vari…
A consequence of Ornstein theory is that the infinite entropy flows associated with Poisson processes and continuous-time irreducible Markov chains on a finite number of states are isomorphic as measure-preserving systems. We give an elementary construction of such an isomorphism that has an additional finitariness property, subject to the additional conditions that the Markov chain has a uniform holding rate and a mixing skeleton.
The aim of this note is to present an elementary proof of a variation of Harris' ergodic theorem of Markov chains. This theorem, dating back to the fifties essentially states that a Markov chain is uniquely ergodic if it admits a ``small'' set which is visited infinitely often. This gives an extension of the ideas of Doeblin to the unbounded state space setting. Often this is established by finding a Lyapunov function with ``small'' level sets. This topic has been studied by many authors (cf. Harris, Hasminskii, Nummelin, Meyn and Tweedie). If the Lyapunov function is strong enough, one has a …
Consider a system of \(n\) players in which each initially starts on a different team. At each time step, we select an individual winner and an individual loser randomly and the loser joins the winner's team. The resulting Markov chain and stochastic matrix clearly have one absorbing state, in which all players are on the same team, but the combinatorics along the way are surprisingly elegant. The expected number of time steps until each team is eliminated is a ratio of binomial coefficients. When a team is eliminated, the probabilities that the players are configured in various partitions of …
A transition matrix $[U_{i,j}]_{i,j\geq 0}$ on $\mathbb{N}$ is said to be almost upper triangular if $U_{i,j}\geq 0\Rightarrow j\geq i-1$, so that the increments of the corresponding Markov chains are at least $-1$; a transition matrix $[L_{i,j}]_{i,j\geq 0}$ is said to be almost lower triangular if $L_{i,j}\geq 0\Rightarrow j\leq i+1$, and then, the increments of the corresponding Markov chains are at most $+1$. In the present paper, we characterize the recurrence, positive recurrence and invariant distribution for the class of almost triangular transition matrices. The upper case appears to …
We introduce an efficient nonreversible Markov chain Monte Carlo algorithm to generate self-avoiding walks with a variable endpoint. In two dimensions, the new algorithm slightly outperforms the two-move nonreversible Berretti-Sokal algorithm introduced by H.~Hu, X.~Chen, and Y.~Deng in \cite{old}, while for three-dimensional walks, it is 3--5 times faster. The new algorithm introduces nonreversible Markov chains that obey global balance and allows for three types of elementary moves on the existing self-avoiding walk: shorten, extend or alter conformation without changing the walk's length.
The recent study by B. De Bruyne, S. N. Majumdar, H. Orland and G. Schehr [arXiv:2110.07573], concerning the conditioning of the Brownian motion and of random walks on global dynamical constraints over a finite time-window $T$, is reformulated as a general framework for the 'microcanonical conditioning' of Markov processes on time-additive observables. This formalism is applied to various types of Markov processes, namely discrete-time Markov chains, continuous-time Markov jump processes and diffusion processes in arbitrary dimension. In each setting, the time-additive observable is also fully…
Inverse problem in the conditioning of Markov processes on trajectory observables : what canonical conditionings can connect two given Markov generators ?
This paper is a survey of various proofs of the so called {\em fundamental theorem of Markov chains}: every ergodic Markov chain has a unique positive stationary distribution and the chain attains this distribution in the limit independent of the initial distribution the chain started with. As Markov chains are stochastic processes, it is natural to use probability based arguments for proofs. At the same time, the dynamics of a Markov chain is completely captured by its initial distribution, which is a vector, and its transition probability matrix. Therefore, arguments based on matrix analysis…
We introduce discrete time Markov chains that preserve uniform measures on boxed plane partitions. Elementary Markov steps change the size of the box from (a x b x c) to ((a-1) x (b+1) x c) or ((a+1) x (b-1) x c). Algorithmic realization of each step involves O((a+b)c) operations. One application is an efficient perfect random sampling algorithm for uniformly distributed boxed plane partitions. Trajectories of our Markov chains can be viewed as random point configurations in the three-dimensional lattice. We compute the bulk limits of the correlation functions of the resulting random point pro…
Starting from elementary considerations about independence and Markov processes in classical probability we arrive at the new concept of conditional monotone independence (or operator-valued monotone independence). With the help of product systems of Hilbert modules we show that monotone conditional independence arises naturally in dilation theory.
