We examine data-processing of Markov chains through the lens of information geometry. We first establish a theory of congruent Markov morphisms within the framework of stochastic matrices. Specifically, we introduce and justify the concept of a linear right inverse (congruent embedding) for lumping, a well-known operation used in Markov chains to extract coarse information. Furthermore, we inspect information projections onto geodesically convex sets of stochastic matrices, and show that under some conditions, projecting (m-projection) onto doubly convex submanifolds can be regarded as a form of data-processing. Finally, we show that the family of lumpable stochastic matrices can be meaningfully endowed with the structure of a foliated manifold and motivate our construction in the context of embedded models and inference.
In this paper we find nonasymptotic exponential upper bounds for the deviation in the ergodic theorem for families of homogeneous Markov processes. We find some sufficient conditions for geometric ergodicity uniformly over a parametric family. We apply this property to the nonasymptotic nonparametric estimation problem for ergodic diffusion processes.
The aim of this note is to investigate the concentration properties of unbounded functions of geometrically ergodic Markov chains. We derive concentration properties of centered functions with respect to the square of the Lyapunov's function in the drift condition satisfied by the Markov chain. We apply the new exponential inequalities to derive confidence intervals for MCMC algorithms. Quantitative error bounds are providing for the regenerative Metropolis algorithm of [5].
We state and prove a quantitative version of the bounded difference inequality for geometrically ergodic Markov chains. Our proof uses the same martingale decomposition as \cite{MR3407208} but, compared to this paper, the exact coupling argument is modified to fill a gap between the strongly aperiodic case and the general aperiodic case.
We consider the problem of testing the identity of a reversible Markov chain against a reference from a single trajectory of observations. Employing the recently introduced notion of a lumping-congruent Markov embedding, we show that, at least in a mildly restricted setting, testing identity to a reversible chain reduces to testing to a symmetric chain over a larger state space and recover state-of-the-art sample complexity for the problem.
We prove that an irreducible aperiodic Markov chain is geometrically ergodic if and only if any separately bounded functional of the stationary chain satisfies an appropriate subgaussian deviation inequality from its mean.
We generalize results of Fan and Zhang [6] on absolute continuity and singularity of the golden Markov geometric series to nonuniform stochastic series given by arbitrary Markov process. In addition we describe an application of these results in fractal geometry.
Dependence coefficients have been widely studied for Markov processes defined by a set of transition probabilities and an initial distribution. This work clarifies some aspects of the theory of dependence structure of Markov chains generated by copulas that are useful in time series econometrics and other applied fields. The main aim of this paper is to clarify the relationship between the notions of geometric ergodicity and geometric ρ-mixing; namely, to point out that for a large number of well known copulas, such as Clayton, Gumbel or Student, these notions are equivalent. Some of the resul…
We provide explicit expressions for the constants involved in the characterisation of ergodicity of sub-geometric Markov chains. The constants are determined in terms of those appearing in the assumed drift and one-step minorisation conditions. The result is fundamental for the study of some algorithms where uniform bounds for these constants are needed for a family of Markov kernels. Our result accommodates also some classes of inhomogeneous chains.
This paper studies the augmented truncation of discrete-time block-monotone Markov chains under geometric drift conditions. We first present a bound for the total variation distance between the stationary distributions of an original Markov chain and its augmented truncation. We also obtain such error bounds for more general cases where an original Markov chain itself may not be block-monotone but is block-wise dominated by a block-monotone Markov chain. Finally we discuss the application of our results to GI/G/1-type Markov chains.
General characterizations of ergodic Markov chains have been developed in considerable detail. In this paper, we study the transience for discrete-time Markov chains on general state spaces, including the geometric transience and algebraic transience. Criteria are presented through establishing the drift condition and considering the first return time. As an application, we give explicit criteria for the random walk on the half line and the skip-free chain on nonnegative integers.
Consider a discrete time, ergodic Markov chain with finite state space which is started from stationarity. Fill and Lyzinski (2014) showed that, in some cases, the hitting time for a given state may be represented as a sum of a geometric number of IID random variables. We extend this result by giving explicit bounds on the distance between any such hitting time and an appropriately chosen geometric sum, along with other related approximations. The compounding random variable in our approximating geometric sum is a strong stationary time for the underlying Markov chain; we also discuss the appr…
We consider Markov chain with spectral gap in $L^2$ space. Assume that $f$ is a bounded function. Then the probabilities of large deviations of average along trajectory satisfy Hoeffding's-type inequalities. These bounds depend only on the stationary mean, spectral gap and the end-points of support of $f$.
In this paper, new conditions for the stability of V-geometrically ergodic Markov chains are introduced. The results are based on an extension of the standard perturbation theory formulated by Keller and Liverani. The continuity and higher regularity properties are investigated. As an illustration, an asymptotic expansion of the invariant probability measure for an autoregressive model with i.i.d. noises (with a non-standard probability density function) is obtained.
In this paper we prove a sharp quantitative version of the Kendall's Theorem. The Kendal Theorem states that under some mild conditions imposed on a probability distribution on positive integers (i.e. probabilistic sequence) one can prove convergence of its renewal sequence. Due to the well-known property - the first entrance last exit decomposition - such results are of interest in the stability theory of time homogeneous Markov chains. In particular the approach may be used to measure rates of convergence of geometrically ergodic Markov chains and consequently implies estimates on convergenc…
This paper considers an approximation usually used when implementing Ramaswami's recursion for the stationary distribution of the M/G/1-type Markov chain. The approximation is called the level-increment-truncation approximation because it truncates level increment at a given threshold. The main contribution of this paper is to present a geometric convergence formula of the level-wise difference between the respective stationary distributions of the original M/G/1-type Markov chain and its LI truncation approximation under the assumption that the level-increment distribution is light-tailed.
Max-stable processes are central models for spatial extremes. In this paper, we focus on some space-time max-stable models introduced in Embrechts et al. (2016). The processes considered induce discrete-time Markov chains taking values in the space of continuous functions from the unit sphere of $\mathbb{R}^3$ to $(0, \infty)$. We show that these Markov chains are geometrically ergodic. An interesting feature lies in the fact that the state space is not locally compact, making the classical methodology inapplicable. Instead, we use the fact that the state space is Polish and apply results pres…
Using Blackwell's definition of comparing two experiments, a comparison is made with \textit{generalized AG - divergence} measure having one and two scalar parameters. Connection of \textit{generalized AG - divergence} measure with \textit{Fisher measure of information} is also presented. A unified \textit{generalization of AG - divergence}and\textit{Jensen-Shannon divergence measures} is also presented.
We consider the invariant measure of a homogeneous continuous- time Markov process in the quarter-plane. The basic solutions of the global balance equation are the geometric distributions. We first show that the invariant measure can not be a finite linear combination of basic geometric distributions, unless it consists of a single basic geo- metric distribution. Second, we show that a countable linear combina- tion of geometric terms can be an invariant measure only if it consists of pairwise-coupled terms. As a consequence, we obtain a complete characterization of all countable linear combin…
For a relatively large class of well-behaved absorbing (or killed) finite Markov chains, we give detailed quantitative estimates regarding the behavior of the chain before it is absorbed (or killed). Typical examples are random walks on box-like finite subsets of the square lattice $\mathbb Z^d$ absorbed (or killed) at the boundary. The analysis is based on Poincaré, Nash, and Harnack inequalities, moderate growth, and on the notions of John and inner-uniform domains.
Let $(X_n)_{n \in\mathbb{N}}$ be a $V$-geometrically ergodic Markov chain on a measurable space $\mathbb{X}$ with invariant probability distribution $π$. In this paper, we propose a discretization scheme providing a computable sequence $(\widehatπ_k)_{k\ge 1}$ of probability measures which approximates $π$ as $k$ growths to infinity. The probability measure $\widehatπ_k$ is computed from the invariant probability distribution of a finite Markov chain. The convergence rate in total variation of $(\widehatπ_k)_{k\ge 1}$ to $π$ is given. As a result, the specific case of first order autoregressiv…
Let $(X_n)_{n=0}^\infty$ denote a Markov chain on a Polish space that has a stationary distribution $\varpi$. This article concerns upper bounds on the Wasserstein distance between the distribution of $X_n$ and $\varpi$. In particular, an explicit geometric bound on the distance to stationarity is derived using generalized drift and contraction conditions whose parameters vary across the state space. These new types of drift and contraction allow for sharper convergence bounds than the standard versions, whose parameters are constant. Application of the result is illustrated in the context of …
Let $P$ be a Markov kernel on a measurable space $\X$ and let $V:\X\r[1,+\infty)$. This paper provides explicit connections between the $V$-geometric ergodicity of $P$ and that of finite-rank nonnegative sub-Markov kernels $\Pc_k$ approximating $P$. A special attention is paid to obtain an efficient way to specify the convergence rate for $P$ from that of $\Pc_k$ and conversely. Furthermore, explicit bounds are obtained for the total variation distance between the $P$-invariant probability measure and the $\Pc_k$-invariant positive measure. The proofs are based on the Keller-Liverani perturbat…
We give computable bounds on the rate of convergence of the transition probabilities to the stationary distribution for a certain class of geometrically ergodic Markov chains. Our results are different from earlier estimates of Meyn and Tweedie, and from estimates using coupling, although we start from essentially the same assumptions of a drift condition toward a ``small set.'' The estimates show a noticeable improvement on existing results if the Markov chain is reversible with respect to its stationary distribution, and especially so if the chain is also positive. The method of proof uses t…
This article continues our study of Markovian consistency and Markov copulae. In particular, we characterize the weak Markovian consistency for finite Markov chains. We discuss some aspects of dependence between the components of a multivariate Markov chain in the context of weak Markovian consistency and strong Markovian consistency. In this connection, we also introduce and discuss the concept of weak Markov copulae.
We investigate the convergence in distribution of sequential empirical processes of dependent data indexed by a class of functions F. Our technique is suitable for processes that satisfy a multiple mixing condition on a space of functions which differs from the class F. This situation occurs in the case of data arising from dynamical systems or Markov chains, for which the Perron--Frobenius or Markov operator, respectively, has a spectral gap on a restricted space. We provide applications to iterative Lipschitz models that contract on average.
We establish quantitative bounds for rates of convergence and asymptotic variances for iterated conditional sequential Monte Carlo (i-cSMC) Markov chains and associated particle Gibbs samplers. Our main findings are that the essential boundedness of potential functions associated with the i-cSMC algorithm provide necessary and sufficient conditions for the uniform ergodicity of the i-cSMC Markov chain, as well as quantitative bounds on its (uniformly geometric) rate of convergence. Furthermore, we show that the i-cSMC Markov chain cannot even be geometrically ergodic if this essential boundedn…
Measure-free discrete time stochastic processes in Riesz spaces were formulated and studied by Kuo, Labuschagne and Watson. Aspects relating martingales, stopping times, convergence of these processes as well as various decomposition were considered. Here we formulate and study Markov processes in a measure-free Riesz space setting.
We argue that the spectral theory of non-reversible Markov chains may often be more effectively cast within the framework of the naturally associated weighted-$L_\infty$ space $L_\infty^V$, instead of the usual Hilbert space $L_2=L_2(π)$, where $π$ is the invariant measure of the chain. This observation is, in part, based on the following results. A discrete-time Markov chain with values in a general state space is geometrically ergodic if and only if its transition kernel admits a spectral gap in $L_\infty^V$. If the chain is reversible, the same equivalence holds with $L_2$ in place of $L_\i…
A virtual Markov chain (VMC) is a sequence $\{X_N\}_{N=0}^{\infty}$ of Markov chains (MCs) coupled together on the same probability space such that $X_N$ has state space $\{0,1,\ldots, N\}$ and such that removing all instances of $N~+~1$ from the sample path of $X_{N+1}$ results in the sample path of $X_N$ almost surely. In this paper, we prove an exact characterization of the triviality of the $σ$-algebra $\bigcap_{N=0}^{\infty}σ(X_N,X_{N+1},\ldots)$. The main tool for doing this is a decomposition theorem that the $σ$-algebra generated by a VMC is equal to the $σ$-algebra generated by a cert…
