Questions are posed regarding the influence that the column sums of the transition probabilities of a stochastic matrix (with row sums all one) have on the stationary distribution, the mean first passage times and the Kemeny constant of the associated irreducible discrete time Markov chain. Some new relationships, including some inequalities, and partial answers to the questions, are given using a special generalized matrix inverse that has not previously been considered in the literature on Markov chains.
This is a survey of the recent progress and open questions on the structure of the sets of 0-1 and non-negative integer matrices with prescribed row and column sums. We discuss cardinality estimates, the structure of a random matrix from the set, discrete versions of the Brunn-Minkowski inequality and the statistical dependence between row and column sums.
We consider the set Sigma(R,C) of all mxn matrices having 0-1 entries and prescribed row sums R=(r_1, ..., r_m) and column sums C=(c_1, ..., c_n). We prove an asymptotic estimate for the cardinality |Sigma(R, C)| via the solution to a convex optimization problem. We show that if Sigma(R, C) is sufficiently large, then a random matrix D in Sigma(R, C) sampled from the uniform probability measure in Sigma(R,C) with high probability is close to a particular matrix Z=Z(R,C) that maximizes the sum of entropies of entries among all matrices with row sums R, column sums C and entries between 0 and 1.…
We consider a simple regression model where a regressor is composed of order statistics and a noise is Markov-modulated. We introduce an empirical bridge of regression residuals and prove its weak convergence to a centered Gaussian process.
We investigate the spectral distribution of large sample covariance matrices with independent columns and entries in the columns that stem from Markov chains. We characterize the limiting spectral densities by their moments. Correspondingly, the proof is based on a moment method.
We count mxn non-negative integer matrices (contingency tables) with prescribed row and column sums (margins). For a wide class of smooth margins we establish a computationally efficient asymptotic formula approximating the number of matrices within a relative error which approaches 0 as m and n grow.
In this paper, we investigate the functional central limit theorem for stochastic processes associated to partial sums of additive functionals of reversible Markov chains with general spate space, under the normalization standard deviation of partial sums. For this case, we show that the functional central limit theorem is equivalent to the fact that the variance of partial sums is regularly varying with exponent 1 and the partial sums satisfy the CLT. It is also equivalent to the conditional CLT.
Consider a discrete time, ergodic Markov chain with finite state space which is started from stationarity. Fill and Lyzinski (2014) showed that, in some cases, the hitting time for a given state may be represented as a sum of a geometric number of IID random variables. We extend this result by giving explicit bounds on the distance between any such hitting time and an appropriately chosen geometric sum, along with other related approximations. The compounding random variable in our approximating geometric sum is a strong stationary time for the underlying Markov chain; we also discuss the appr…
Let X_{n} be an integer valued Markov Chain with finite state space. Let S_{n}=\sum_{k=0}^{n}X_{k} and let L_{n}(x) be the number of times S_{k} hits x up to step n. Define the normalized local time process t_{n}(x) by t_{n}(x)=\frac{L_{n}(\sqrt{n}(x)}{\sqrt{n}}. The subject of this paper is to prove a functional, weak invariance principle for the normalized sequence t_{n}, i.e. we prove that under some assumptions about the Markov Chain, the normalized local times converge in distribution to the local time of the Brownian Motion.
We consider nonnegative integer matrices with specified row and column sums and upper bounds on the entries. We show that the logarithm of the number of such matrices is approximated by a concave function of the row and column sums. We give efficiently computable estimators for this function, including one suggested by a maximum-entropy random model; we show that these estimators are asymptotically exact as the dimension of the matrices goes to infinity. We finish by showing that, for kappa >= 2 and for sufficiently small row and column sums, the number of matrices with these row and column su…
Stochastically monotone Markov chains arise in many applied domains, especially in the setting of queues and storage systems. Poisson's equation is a key tool for analyzing additive functionals of such models, such as cumulative sums of waiting times or sums of rewards. In this paper, we show that when the reward function for such a Markov chain is monotone, the solution of Poisson's equation is monotone. This implies that the value function associated with infinite horizon average reward is monotone in the state when the reward is monotone.
In this paper we consider stationary Markov chains with trivial two-sided tail sigma field, and prove that additive functionals satisfy the central limit theorem provided the variance of partial sums divided by n is bounded.
The Laplace transform of partial sums of the square of a non-centered Gauss-Markov process, conditioning on its starting point, is explicitly computed. The parameters of multiplicative ergodicity are deduced.
In this paper, we present reversibility preserving operations on Markov chain transition matrices. Simple row and column operations allow us to create new reversible transition matrices and yield an easy method for checking a Markov chain for reversibility.
In this paper we study the additive functionals of Markov chains via conditioning with respect to both past and future of the chain. We shall point out new sufficient projective conditions, which assure that the variance of partial sums of n consecutive random variables of a stationary Markov chain is linear in n. The paper also addresses the central limit theorem problem and is listing several open questions.
In this paper we obtain the central limit theorem for triangular arrays of non-homogeneous Markov chains under a condition imposed to the maximal coefficient of correlation. The proofs are based on martingale techniques and a sharp lower bound estimate for the variance of partial sums. The results complement an important central limit theorem of Dobrushin based on the contraction coefficient.
We build optimal exponential bounds for the probabilities of large deviations of sums \sum_{k=1}^nf(X_k) where (X_k) is a finite reversible Markov chain and f is an arbitrary bounded function. These bounds depend only on the stationary mean E_πf, the end-points of the support of f, the sample size n and the second largest eigenvalue λof the transition matrix.
Let $C\geq 2$ be a positive integer. Consider the set of $n\times n$ non-negative integer matrices whose row sums and column sums are all equal to $Cn$ and let $X=(X_{ij})_{1\leq i,j\leq n}$ be uniformly distributed on this set. This $X$ is called the random contingency table with uniform margin. In this paper, we study various asymptotic properties of $X=(X_{ij})_{1\leq i,j\leq n}$ as $n\to\infty$.
We present a randomized approximation algorithm for counting contingency tables, mxn non-negative integer matrices with given row sums R=(r_1, ..., r_m) and column sums C=(c_1, ..., c_n). We define smooth margins (R,C) in terms of the typical table and prove that for such margins the algorithm has quasi-polynomial N^{O(ln N)} complexity, where N=r_1+...+r_m=c_1+...+c_n. Various classes of margins are smooth, e.g., when m=O(n), n=O(m) and the ratios between the largest and the smallest row sums as well as between the largest and the smallest column sums are strictly smaller than the golden rati…
We consider sums of increments given by a functional of a stationary Markov chain. Letting $T$ be the first return time of the partial sums process to $(-\infty,0]$, under general assumptions, we determine the asymptotic behavior of the survival probability, $\mathbb{P}(T\ge t)\sim Ct^{-1/2}$ for an explicit constant $C$. Our analysis is based on a connection between the survival probability and the running maximum of the time-reversed process, and relies on a functional central limit theorem for Markov chains. Our result extends the classic theorem of Sparre Anderson on sums of mean zero and …
The solution of Poisson's equation plays a key role in constructing the martingale through which sums of Markov correlated random variables can be analyzed. In this paper, we study two different representations for the solution in countable state space, one based on regenerative structure and the other based on an infinite sum of expectations. We also consider integrability and related uniqueness issues associated with solutions to Poisson's equation, and provide verifiable Lyapunov conditions to support our theory. Our key results include a central limit theorem and law of the iterated logari…
Consider the partial sums {S_t} of a real-valued functional F(Phi(t)) of a Markov chain {Phi(t)} with values in a general state space. Assuming only that the Markov chain is geometrically ergodic and that the functional F is bounded, the following conclusions are obtained: 1. Spectral theory: Well-behaved solutions can be constructed for the ``multiplicative Poisson equation''. 2. A ``multiplicative'' mean ergodic theorem: For all complex αin a neighborhood of the origin, the normalized mean of \exp(αS_t) converges exponentially fast to a solution of the multiplicative Poisson equation. 3. Edg…
We introduce the cluster index of a multivariate regularly varying stationary sequence and characterize the index in terms of the spectral tail process. This index plays a major role in limit theory for partial sums of regularly varying sequences. We illustrate the use of the cluster index by characterizing infinite variance stable limit distributions and precise large deviation results for sums of multivariate functions acting on a stationary Markov chain under a drift condition.
We consider a large class of piecewise expanding maps T of [0,1] with a neutral fixed point, and their associated Markov chain Y_i whose transition kernel is the Perron-Frobenius operator of T with respect to the absolutely continuous invariant probability measure. We give a large class of unbounded functions f for which the partial sums of f\circ T^i satisfy both a central limit theorem and a bounded law of the iterated logarithm. For the same class, we prove that the partial sums of f(Y_i) satisfy a strong invariance principle. When the class is larger, so that the partial sums of f\circ T^i…
We propose some backward-forward martingale decompositions for functions of reversible Markov chains. These decompositions are used to prove the functional CLT for reversible Markov chains with asymptotically linear variance of partial sums. We also provide a proof of the equivalence between asymptotic linearity of the variance and convergence of the integral of $1/(1-t)$ with respect to the associated spectral measure $ρ$. We also study the asymptotic behavior of linear processes having as innovations mean zero square integrable functions of stationary reversible Markov chains. We apply this …
Using the renewal approach we prove exponential inequalities for additive functionals and empirical processes of ergodic Markov chains, thus obtaining counterparts of inequalities for sums of independent random variables. The inequalities do not require functions of the chain to be bounded and moreover all the involved constants are given by explicit formulas whenever the usual drift condition holds, which may be of interest in practical applications e.g. to MCMC algorithms.
In this paper, we analyze the stochastic properties of some large size (area) polyominoe's perimeter such that the directed column-convex polyomino, the column-convex polyomino, the directed diagonally-convex polyomino, the staircase (or parallelogram) polyomino, the escalier polyomino, the wall (or bargraph) polyomino. All polyominoes considered here are made of contiguous, not-empty columns, without holes, such that each column must be adjacent to some cell of the previous column. We compute the asymptotic (for large size $n$) Gaussian distribution of the perimeter, including the correspondi…
This paper proposes a new algorithm for computing the stationary distribution vector in continuous-time upper block-Hessenberg Markov chains. To this end, we consider the last-block-column-linearly-augmented (LBCL-augmented) truncation of the (infinitesimal) generator of the upper block-Hessenberg Markov chain. The LBCL-augmented truncation is a linearly-augmented truncation such that the augmentation distribution has its probability mass only on the last block column. We first derive an upper bound for the total variation distance between the respective stationary distribution vectors of the …
We provide deviation inequalities for properly normalized sums of bifurcating Markov chains on Galton-Watson tree. These processes are extension of bifurcating Markov chains (which was introduced by Guyon to detect cellular aging from cell lineage) in case the index set is a binary Galton-Watson process. As application, we derive deviation inequalities for the least-squares estimator of autoregressive parameters of bifurcating autoregressive processes with missing data. These processes allow, in case of cell division, to take into account the cell's death. The results are obtained under an uni…
We prove deviation bounds for the random variable $\sum_{i=1}^{n} f_i(Y_i)$ in which $\{Y_i\}_{i=1}^{\infty}$ is a Markov chain with stationary distribution and state space $[N]$, and $f_i: [N] \rightarrow [-a_i, a_i]$. Our bound improves upon previously known bounds in that the dependence is on $\sqrt{a_1^2+\cdots+a_n^2}$ rather than $\max_{i}\{a_i\}\sqrt{n}.$ We also prove deviation bounds for certain types of sums of vector--valued random variables obtained from a Markov chain in a similar manner. One application includes bounding the expected value of the Schatten $\infty$-norm of a random…
For a Harris ergodic Markov chain $(X_n)_{n\ge 0}$, on a general state space, started from the so called small measure or from the stationary distribution we provide optimal estimates for Orlicz norms of sums $\sum_{i=0}^τf(X_i)$, where $τ$ is the first regeneration time of the chain. The estimates are expressed in terms of other Orlicz norms of the function $f$ (wrt the stationary distribution) and the regeneration time $τ$ (wrt the small measure). We provide applications to tail estimates for additive functionals of the chain $(X_n)$ generated by unbounded functions as well as to classical l…
We study a multivariate Markov chain on the symmetric group with remarkable enumerative properties. We conjecture that the stationary distribution of this Markov chain can be expressed in terms of positive sums of Schubert polynomials. This Markov chain is a multivariate generalization of a Markov chain introduced by the first author in the study of random affine Weyl group elements.
This paper studies the last-column-block-augmented northwest-corner truncation (LC-block-augmented truncation, for short) of discrete-time block-monotone Markov chains under subgeometric drift conditions. The main result of this paper is to present an upper bound for the total variation distance between the stationary probability vectors of a block-monotone Markov chain and its LC-block-augmented truncation. The main result is extended to Markov chains that themselves may not be block monotone but are block-wise dominated by block-monotone Markov chains satisfying modified drift conditions. Fi…
In this article we develop a general method by which one can explicitly evaluate certain sums of $n$-th powers of products of $d\geq 1$ elementary trigonometric functions evaluated at $\mathbf{m}=(m_1,\ldots,m_d)$-th roots of unity. Our approach is to first identify the individual terms in the expression under consideration as eigenvalues of a discrete Laplace operator associated to a graph whose vertices form a $d$-dimensional discrete torus $G_{\mathbf{m}}$ which depends on $\mathbf{m}$. The sums in question are then related to the $n$-th step of a Markov chain on $G_{\mathbf{m}}$. The Marko…
Using the renewal approach we prove Bernstein-like inequalities for additive functionals of geometrically ergodic Markov chains, thus obtaining counterparts of inequalities for sums of independent random variables. The coefficient in the sub-Gaussian part of our estimate is the asymptotic variance of the additive functional, i.e. the variance of the limiting Gaussian variable in the Central Limit Theorem for Markov chains. This refines earlier results by R. Adamczak and W. Bednorz, which were obtained under the additional assumption of strong aperiodicity of the chain.
We obtain large deviations theorems for nonconventional sums with underlying process being a Markov process satisfying the Doeblin condition or a dynamical system such as subshift of finite type or hyperbolic or expanding transformation.
We give an algorithm that generates a uniformly random contingency table with specified marginals, i.e. a matrix with non-negative integer values and specified row and column sums. Such algorithms are useful in statistics and combinatorics. When $Δ^4< M/5$, where $Δ$ is the maximum of the row and column sums and $M$ is the sum of all entries of the matrix, our algorithm runs in time linear in $M$ in expectation. Most previously published algorithms for this problem are approximate samplers based on Markov chain Monte Carlo, whose provable bounds on the mixing time are typically polynomials wit…
The famous results of Komlós, Major and Tusnády (see [15] and [17]) state that it is possible to approximate almost surely the partial sums of size n of i.i.d. centered random variables in L p (p > 2) by a Wiener process with an error term of order o(n 1/p). Very recently, Berkes, Liu and Wu [3] extended this famous result to partial sums associated with functions of an i.i.d. sequence, provided a condition on a functional dependence measure in L p is satisfied. In this paper, we adapt the method of Berkes, Liu and Wu to partial sums of functions of random iterates. Taking advantage of the Mar…
We provide a many-to-few formula in the general setting of non-local branching Markov processes. This formula allows one to compute expectations of k-fold sums over functions of the population at k different times. The result generalises [14] to the non-local setting, as introduced in [11] and [8]. As an application, we consider the case when the branching process is critical, and conditioned to survive for a large time. In this setting, we prove a general formula for the limiting law of the death time of the most recent common ancestor of two particles selected uniformly from the population a…
This paper develops an optimal Chernoff type bound for the probabilities of large deviations of sums $\sum_{k=1}^n f (X_k)$ where $f$ is a real-valued function and $(X_k)_{k \in \mathbb{Z}_{\ge 0}}$ is a finite state Markov chain with an arbitrary initial distribution and an irreducible transition probability matrix satisfying a mild assumption on its positivity pattern, related to the function $f$ being considered. The novelty lies in this being a non-asymptotic finite sample bound. Further, our bound is optimal in the large deviations sense, attaining a constant prefactor and an exponential …
We study the Markov chain on $\mathbf{F}_p$ obtained by applying a function $f$ and adding $\pmγ$ with equal probability. When $f$ is a linear function, this is the well-studied Chung--Diaconis--Graham process. We consider two cases: when $f$ is the extension of a rational function which is bijective, and when $f(x)=x^2$. In the latter case, the stationary distribution is not uniform and we characterize it when $p=3\pmod{4}$. In both cases, we give an almost linear bound on the mixing time, showing that the deterministic function dramatically speeds up mixing. The proofs involve establishing b…
In this paper we study the functional central limit theorem for stationary Markov chains with self-adjoint operator and general state space. We investigate the case when the variance of the partial sum is not asymptotically linear in n; and establish that conditional convergence in distribution of partial sums implies functional CLT. The main tools are maximal inequalities that are further exploited to derive conditions for tightness and convergence to the Brownian motion.
The sums and maxima of weighted non-stationary random length sequences of regularly varying random variables may have the same tail and extremal indices, Markovich and Rodionov (2020). The main constraints are that there exists a unique series in a scheme of series with the minimum tail index, the tail of the term number is lighter than the tail of the terms and the weights are positive constants. These assumptions are changed here: a bounded random number of series is allowed to have the minimum tail index, the tail of the term number may be heavier than the tail of the terms and the weights …
For the random interval partition of $[0,1]$ generated by the uniform stick-breaking scheme known as GEM$(1)$, let $u_k$ be the probability that the first $k$ intervals created by the stick-breaking scheme are also the first $k$ intervals to be discovered in a process of uniform random sampling of points from $[0,1]$. Then $u_k$ is a renewal sequence. We prove that $u_k$ is a rational linear combination of the real numbers $1, ζ(2), \ldots, ζ(k)$ where $ζ$ is the Riemann zeta function, and show that $u_k$ has limit $1/3$ as $k \to \infty$. Related results provide probabilistic interpretations …
We consider triangular arrays of Markov chains that converge weakly to a diffusion process. Edgeworth type expansions of third order for transition densities are proved. This is done for time horizons that converge to 0. For this purpose we represent the transition density as a functional of densities of sums of i.i.d. variables. This will be done by application of the parametrix method. Then we apply Edgeworth expansions to the densities. The resulting series gives our Edgeworth-type expansion for the transition density of Markov chains. The research is motivated by applications to high frequ…
In this paper a method based on a Markov chain Monte Carlo (MCMC) algorithm is proposed to compute the probability of a rare event. The conditional distribution of the underlying process given that the rare event occurs has the probability of the rare event as its normalizing constant. Using the MCMC methodology a Markov chain is simulated, with that conditional distribution as its invariant distribution, and information about the normalizing constant is extracted from its trajectory. The algorithm is described in full generality and applied to the problem of computing the probability that a h…
Bifurcating Markov chains (BMC) are Markov chains indexed by a full binary tree representing the evolution of a trait along a population where each individual has two children. We provide a central limit theorem for general additive functionals of BMC, and prove the existence of three regimes. This corresponds to a competition between the reproducing rate (each individual has two children) and the ergodicity rate for the evolution of the trait. This is in contrast with the work of Guyon (2007), where the considered additive functionals are sums of martingale increments, and only one regime app…
We prove a new concentration inequality for U-statistics of order two for uniformly ergodic Markov chains. Working with bounded and $π$-canonical kernels, we show that we can recover the convergence rate of Arcones and Gin{é} who proved a concentration result for U-statistics of independent random variables and canonical kernels. Our result allows for a dependence of the kernels $h_{i,j}$ with the indexes in the sums, which prevents the use of standard blocking tools. Our proof relies on an inductive analysis where we use martingale techniques, uniform ergodicity, Nummelin splitting and Bernst…
