Using elementary methods, we prove that for a countable Markov chain $P$ of ergodic degree $d > 0$ the rate of convergence towards the stationary distribution is subgeometric of order $n^{-d}$, provided the initial distribution satisfies certain conditions of asymptotic decay. An example, modelling a renewal process and providing a markovian approximation scheme in dynamical system theory, is worked out in detail, illustrating the relationships between convergence behaviour, analytic properties of the generating functions associated to transition probabilities and spectral properties of the Markov operator $P$ on the Banach space $\ell_1$. Explicit conditions allowing to obtain the actual asymptotics for the rate of convergence are also discussed.
Let $σ:\boldsymbolΣ\to\boldsymbolΣ$ be the left shift acting on $ \boldsymbolΣ $, a one-sided Markov subshift on a countable alphabet. Our intention is to guarantee the existence of $σ$-invariant Borel probabilities that maximize the integral of a given locally Hölder continuous potential $ A : \boldsymbolΣ \to \mathbb R $. Under certain conditions, we are able to show not only that $A$-maximizing probabilities do exist, but also that they are characterized by the fact their support lies actually in a particular Markov subshift on a finite alphabet. To that end, we make use of objects dual to …
This work focuses on a class of regime-switching jump diffusion processes, in which the switching component has countably infinite many states or regimes. The existence and uniqueness of the underlying process are obtained by an interlacing procedure. Then the Feller and strong Feller properties of such processes are derived by the coupling method and an appropriate Radon-Nikodym derivative. Finally the paper studies exponential ergodicity of regime-switching jump-diffusion processes.
We consider a large family of discrete and continuous time controlled Markov processes and study an ergodic risk-sensitive minimization problem. Under a blanket stability assumption, we provide a complete analysis to this problem. In particular, we establish uniqueness of the value function and verification result for optimal stationary Markov controls, in addition to the existence results. We also revisit this problem under a near-monotonicity condition but without any stability hypothesis. Our results also include policy improvement algorithms both in discrete and continuous time frameworks.
We begin by giving a new proof of the equivalence between the Liouville property and vanishing of the drift for symmetric random walks with finite first moments on finitely generated groups; a result which was first established by Kaimanovich-Vershik and Karlsson-Ledrappier. We then proceed to prove that the product of the Poisson boundary of any countable measured group $(G,μ)$ with any ergodic $(G,\checkμ)$-space is still ergodic, which in particular yields a new proof of weak mixing for the double Poisson boundary of $(G,μ)$ when $μ$ is symmetric. Finally, we characterize the failure of wea…
We establish the large deviation principle for a topological Markov shift over infinite alphabet which satisfies strong combinatorial assumptions called ``finite irreducibility'' or ``finite primitiveness''. More precisely, we assume the existence of a Gibbs state for a potential $ϕ$ in the sense of Bowen, and prove the level-2 Large Deviation Principles for the distribution of empirical means under the Gibbs state, as well as that of weighted periodic points and iterated pre-images. The rate function is written with the pressure and the free energy associated with the potential $ϕ$.
We consider ergodic $\mathrm{Sym}(\mathbb{N})$-invariant probability measures on the space of $L$-structures with domain $\mathbb{N}$ (for $L$ a countable relational language), and call such a measure a properly ergodic structure when no isomorphism class of structures is assigned measure $1$. We characterize those theories in countable fragments of $\mathcal{L}_{ω_1, ω}$ for which there is a properly ergodic structure concentrated on the models of the theory. We show that for a countable fragment $F$ of $\mathcal{L}_{ω_1, ω}$ the almost-sure $F$-theory of a properly ergodic structure has cont…
Previous work showed that all Bernoulli shifts over a free group are orbit-equivalent. This result is strengthened here by replacing Bernoulli shifts with the wider class of properly ergodic countable state Markov chains over a free group. A list of related open problems is provided.
For a countable-state Markov decision process we introduce an embedding which produces a finite-state Markov decision process. The finite-state embedded process has the same optimal cost, and moreover, it has the same dynamics as the original process when restricting to the approximating set. The embedded process can be used as an approximation which, being finite, is more convenient for computation and implementation.
Here we propose the Donsker-Varadhan-type compactness conditions and prove the joint large deviation principle for the empirical measure and empirical flow of Markov renewal processes (semi-Markov processes) with a countable state space, generalizing the relevant results for continuous-time Markov chains with a countable state space obtained in [Ann. Inst. H. Poincaré Probab. Statist. 51, 867-900 (2015)] and [Stoch. Proc. Appl. 125, 2786-2819 (2015)], as well as the relevant results for Markov renewal processes with a finite state space obtained in [Adv. Appl. Probab. 48, 648-671 (2016)]. In p…
It is known that the Dobrushin's ergodicity coefficient is one of the effective tools to study a behavior of non-homogeneous Markov chains. In the present paper, we define such an ergodicity coefficient of a positive mapping defined on ordered Banach space with a base (OBSB), and study its properties. In terms of this coefficient we prove the equivalence uniform and weak ergodicities of homogeneous Markov chains. This extends earlier results obtained in case of von Neumann algebras. Such a result allowed to establish a category theorem for uniformly ergodic Markov operators. We find necessary …
We consider killed Markov decision processes for countable models on a finite time-interval. Existence of a uniform $\varepsilon$-optimal policy is proven. We show the correctness of the fundamental equation. The optimal control problem is reduced to a similar problem for the derived model. We receive an optimality equation and a method for the construction of simple optimal policies. The sufficiency of simple policies for countable models is proven. We show the correctness of the Markovian property. Additionally, a dynamic programming principle is considered.
We prove that an irreducible aperiodic Markov chain is geometrically ergodic if and only if any separately bounded functional of the stationary chain satisfies an appropriate subgaussian deviation inequality from its mean.
This paper is devoted to solving a time-inconsistent risk-sensitive control problem with parameter $\e$ and its limit case ($\e\rightarrow0^+$) for countable-stated Markov decision processes (MDPs for short). Since the cost functional is time-inconsistent, it is impossible to find a global optimal strategy for both cases. Instead, for each case, we will prove the existence of time-inconstant equilibrium strategies which verify the so-called step-optimality. Moreover, we prove the convergence of $\e$-equilibriums and the corresponding value functions as $\e\rightarrow0^+$.
We study ergodic properties of nonlinear Markov chains and stochastic McKean-Vlasov equations. For nonlinear Markov chains we obtain sufficient conditions for existence and uniqueness of an invariant measure and uniform ergodicity. We also prove optimality of these conditions. For stochastic McKean-Vlasov equations we establish exponential convergence of their solutions to stationarity in the total variation metric under Veretennikov-Khasminskii-type conditions.
Ergodicity is a fundamental issue for a stochastic process. In this paper, we refine results on ergodicity for a general type of Markov chain to a specific type or the $GI/G/1$-type Markov chain, which has many interesting and important applications in various areas. It is of interest to obtain conditions in terms of system parameters or the given information about the process, under which the chain has various ergodic properties. Specifically, we provide necessary and sufficient conditions for geometric, strong and polynomial ergodicity, respectively.
Partially exchangeable sequences representable as mixtures of Markov chains are completely specified by de Finetti's mixing measure. The paper characterizes, in terms of a subclass of hidden Markov models, the partially exchangeable sequences with mixing measure concentrated on a countable set, both for discrete valued and for Polish valued sequences.
We present Lyapunov-type conditions for non-strong ergodicity of Markov processes. Some concrete models are discussed including diffusion processes on Riemannian manifolds and Ornstein-Uhlenbeck processes driven by symmetric $α$-stable processes. For SDE driven by $α$-stable process ($α\in (0,2]$) with polynomial drift, the strong ergodicity or not is independent on $α$.
We establish verifiable general sufficient conditions for exponential or subexponential ergodicity of Markov processes that may lack the strong Feller property. We apply the obtained results to show exponential ergodicity of a variety of nonlinear stochastic partial differential equations with additive forcing, including 2D stochastic Navier-Stokes equations. Our main tool is a new version of the generalized coupling method.
In our earlier paper, a generalized Dobrushin ergodicity coefficient of Markov operators (acting on abstract state spaces) with respect to a projection $P$, has been introduced and studied. It turned out that the introduced coefficient was more effective than the usual ergodicity coefficient. In the present work, by means of a left consistent Markov projections and the generalized Dobrushin's ergodicity coefficient, we investigate uniform and weak $P$-ergodicities of non-homogeneous discrete Markov chains (NDMC) on abstract state spaces. It is easy to show that uniform $P$-ergodicity implies a…
Let $(ξ_n)_{n=0}^\infty$ be a nonhomogeneous Markov chain taking values from finite state-space of $\mathbf{X}=\{1,2,\ldots,b\}$. In this paper, we will study the generalized entropy ergodic theorem with almost-everywhere and $\mathcal{L}_1$ convergence for nonhomogeneous Markov chains, which generalizes the corresponding classical results for the Markov chains.
We prove the convergence at an exponential rate towards the invariant probability measure for a class of solutions of stochastic differential equations with finite delay. This is done, in this non-Markovian setting, using the cluster expansion method, inspired from [4] or [14]. As a consequence, the results hold for small perturbations of ergodic diffusions.
In this paper, we give quantitative bounds on the $f$-total variation distance from convergence of an Harris recurrent Markov chain on an arbitrary under drift and minorisation conditions implying ergodicity at a sub-geometric rate. These bounds are then specialized to the stochastically monotone case, covering the case where there is no minimal reachable element. The results are illustrated on two examples from queueing theory and Markov Chain Monte Carlo.
We discuss the ergodic properties of quasi-Markovian stochastic differential equations, providing general conditions that ensure existence and uniqueness of a smooth invariant distribution and exponential convergence of the evolution operator in suitably weighted $L^{\infty}$ spaces, which implies the validity of central limit theorem for the respective solution processes. The main new result is an ergodicity condition for the generalized Langevin equation with configuration-dependent noise and (non-)conservative force.
We present formulas for quasi-ergodic limits of finite absorbing Markov chains. Since the irreducible case has been solved in 1965 by Darroch and Seneta, we focus on the reducible case, and our results are based on a very precise asymptotic analysis of the (exponential and polynomial) growth behaviour along admissible paths
We give a bound on the mixing time of a uniformly ergodic, reversible Markov chain in terms of the spectral radius of the transition operator. This bound has been established previously in finite state spaces, and is widely believed to hold in general state spaces, but a proof has not been provided to our knowledge.
In this paper the stability and the perturbation bounds of Markov operators acting on abstract state spaces are investigated. Here, an abstract state space is an ordered Banach space where the norm has an additivity property on the cone of positive elements. We basically study uniform ergodic properties of Markov operators by means of so-called a generalized Dobrushin's ergodicity coefficient. This allows us to get several convergence results with rates. Some results on quasi-compactness of Markov operators are proved in terms of the ergodicity coefficient. Furthermore, a characterization of u…
We study ergodic properties of a class of Markov-modulated general birth-death processes under fast regime switching. The first set of results concerns the ergodic properties of the properly scaled joint Markov process with a parameter that is taken large. Under very weak hypotheses, we show that if the averaged process is exponentially ergodic for large values of the parameter, then the same applies to the original joint Markov process. The second set of results concerns steady-state diffusion approximations, under the assumption that the 'averaged' fluid limit exists. Here, we establish conv…
We consider Markov chain with spectral gap in $L^2$ space. Assume that $f$ is a bounded function. Then the probabilities of large deviations of average along trajectory satisfy Hoeffding's-type inequalities. These bounds depend only on the stationary mean, spectral gap and the end-points of support of $f$.
We consider a time inhomogeneous strong Markov process $(ξ_t)_{t\ge 0}$ taking values in a Polish state space whose semigroup has a $T$-periodic structure. We give simple conditions which imply ergodicity of the grid chain $(ξ_{kT})_{k\in \mathbb{N}_0}$ In case of $d$-dimensional possibly degenerate diffusions, the conditions are stated in terms of drift and diffusion coefficient of the process.
Our purpose is to prove central limit theorem for countable nonhomogeneous Markov chain under the condition of uniform convergence of transition probability matrices for countable nonhomogeneous Markov chain in Cesàro sense. Furthermore, we obtain a corresponding moderate deviation theorem for countable nonhomogeneous Markov chain by Gärtner-Ellis theorem and exponential equivalent method.
We study perturbation theory and uniform ergodicity for discrete-time Markov chains on general state spaces in terms of the uniform moments of the first hitting times on some set. The methods we adopt are different from previous ones. For reversible and non-negative definite Markov chains, we first investigate the geometrically ergodic convergence rates. Based on the estimates, together with a first passage formula, we then get the convergence rates in uniform ergodicity. If the transition kernel $P$ is only reversible, we transfer to study the two-skeleton chain with the transition kernel $P^…
In this paper, the forgetting of the initial distribution for a non-ergodic Hidden Markov Models (HMM) is studied. A new set of conditions is proposed to establish the forgetting property of the filter, which significantly extends all the existing results. Both a pathwise-type convergence of the total variation distance of the filter started from two different initial distributions, and a convergence in expectation are considered. The results are illustrated using generic models of non-ergodic HMM and extend all the results known so far.
We consider ergodic backward stochastic differential equations, in a setting where noise is generated by a countable state uniformly ergodic Markov chain. We show that for Lipschitz drivers such that a comparison theorem holds, these equations admit unique solutions. To obtain this result, we show by coupling and splitting techniques that uniform ergodicity estimates of Markov chains are robust to perturbations of the rate matrix, and that these perturbations correspond in a natural way to EBSDEs. We then consider applications of this theory to Markov decision problems with a risk-averse avera…
In this paper we find nonasymptotic exponential upper bounds for the deviation in the ergodic theorem for families of homogeneous Markov processes. We find some sufficient conditions for geometric ergodicity uniformly over a parametric family. We apply this property to the nonasymptotic nonparametric estimation problem for ergodic diffusion processes.
In this paper, new conditions for the stability of V-geometrically ergodic Markov chains are introduced. The results are based on an extension of the standard perturbation theory formulated by Keller and Liverani. The continuity and higher regularity properties are investigated. As an illustration, an asymptotic expansion of the invariant probability measure for an autoregressive model with i.i.d. noises (with a non-standard probability density function) is obtained.
We study properties of the Laplace transforms of non-negative additive functionals of Markov chains. We are namely interested in a multiplicative ergodicity property used in [18] to study bifurcating processes with ancestral dependence. We develop a general approach based on the use of the operator perturbation method. We apply our general results to two examples of Markov chains, including a linear autoregressive model. In these two examples the operator-type assumptions reduce to some expected finite moment conditions on the functional (no exponential moment conditions are assumed in this wo…
Let $(S_n)_n$ be a $R^d$-valued random walk ($d\geq2$). Using Babillot's method [2], we give general conditions on the characteristic function of $S_n$ under which $(S_n)_n$ satisfies the same renewal theorem as the classical one obtained for random walks with i.i.d. non-centered increments. This statement is applied to additive functionals of strongly ergodic Markov chains under the non-lattice condition and (almost) optimal moment conditions.
New relations between ergodic rate, L_p convergence rates, and asymptotic behavior of tail probabilities for hitting times of a time homogeneous Markov process are established. For L_p convergence rates and related spectral and functional properties (spectral gap and Poincare inequality) sufficient conditions are given in the terms of an exponential ϕ-coupling. This provides sufficient conditions for L_p convergence rates in the terms of appropriate combination of `local mixing' and `recurrence' conditions on the initial process, typical in the ergodic theory of Markov processes. The range of …
We revisit functional central limit theorems for additive functionals of ergodic Markov diffusion processes. Translated in the language of partial differential equations of evolution, they appear as diffusion limits in the asymptotic analysis of Fokker-Planck type equations. We focus on the square integrable framework, and we provide tractable conditions on the infinitesimal generator, including degenerate or anomalously slow diffusions. We take advantage on recent developments in the study of the trend to the equilibrium of ergodic diffusions. We discuss examples and formulate open problems.
In this paper we prove a sharp quantitative version of the Kendall's Theorem. The Kendal Theorem states that under some mild conditions imposed on a probability distribution on positive integers (i.e. probabilistic sequence) one can prove convergence of its renewal sequence. Due to the well-known property - the first entrance last exit decomposition - such results are of interest in the stability theory of time homogeneous Markov chains. In particular the approach may be used to measure rates of convergence of geometrically ergodic Markov chains and consequently implies estimates on convergenc…
In this paper, we give a sufficient condition for the existence of a quasi-ergodic distribution for absorbing Markov processes. Using an orthogonal-polynomial approach, we prove that the previous main result is valid for the birth-death process on the nonnegative integers with 0 an absorbing boundary and $\infty$ an entrance boundary. We also show that the quasi-ergodic distribution is stochastically larger than the unique quasi-stationary distribution in the sense of monotone likelihood-ratio ordering for the birth-death process.
We derive sufficient conditions for subgeometric f-ergodicity of strongly Markovian processes. We first propose a criterion based on modulated moment of some delayed return-time to a petite set. We then formulate a criterion for polynomial f-ergodicity in terms of a drift condition on the generator. Applications to specific processes are considered, including Langevin tempered diffusions on R^n and storage models.
This paper consists of four parts. In the first part, we explain what eigenvalues we are interested in and show the difficulties of the study on the first (non-trivial) eigenvalue through examples. In the second part, we present some (dual) variational formulas and explicit bounds for the first eigenvalue of Laplacian on Riemannian manifolds or Jacobi matrices (Markov chains). Here, a probabilistic approach--the coupling methods is adopted. In the third part, we introduce recent lower bounds of several basic inequalities; these are based on a generalization of Cheeger's approach which comes fr…
We prove that moderate deviations for empirical measures for countable nonhomogeneous Markov chains hold under the assumption of uniform convergence of transition probability matrices for countable nonhomogeneous Markov chains in Cesàro sense.
We are interested in quasi-stationarity and quasi-ergodicity when the absorbing boundary is moving. First we show that, in the moving boundary case, the quasi-stationary distribution and the quasi-limiting distribution are not well-defined when the boundary is oscillating periodically. Then we show the existence of a quasi-ergodic distribution for any discrete-time irreducible Markov chain defined on a finite space state in the fixed boundary case. Finally we use this last result to show the quasi-ergodicity in the moving boundary case.
We show how to map the states of an ergodic Markov chain to Euclidean space so that the squared distance between states is the expected commuting time. We find a minimax characterization of commuting times, and from this we get monotonicity of commuting times with respect to equilibrium transition rates. All of these results are familiar in the case of time-reversible chains, where techniques of classical electrical theory apply. In presenting these results, we take the opportunity to develop Markov chain theory in a `conformally correct' way
This paper provides a general and abstract approach to approximate ergodic regimes of Markov and Feller processes. More precisely, we show that the recursive algorithm presented in Lamberton & Pages (2002) and based on simulation algorithms of stochastic schemes with decreasing step can be used to build invariant measures for general Markov and Feller processes. We also propose applications in three different configurations: Approximation of Markov switching Brownian diffusion ergodic regimes using Euler scheme, approximation of Markov Brownian diffusion ergodic regimes with Milstein scheme an…
Max-stable processes are central models for spatial extremes. In this paper, we focus on some space-time max-stable models introduced in Embrechts et al. (2016). The processes considered induce discrete-time Markov chains taking values in the space of continuous functions from the unit sphere of $\mathbb{R}^3$ to $(0, \infty)$. We show that these Markov chains are geometrically ergodic. An interesting feature lies in the fact that the state space is not locally compact, making the classical methodology inapplicable. Instead, we use the fact that the state space is Polish and apply results pres…
