It is shown that transient graphs for the simple random walk do not admit a nearest neighbor transient Markov chain (not necessarily a reversible one), that crosses all edges with positive probability, while there is such chain for the square grid $\mathbb{Z}^2$. In particular, the $d$-dimensional grid $\mathbb{Z}^d$ admits such a Markov chain only when $d=2$. For $d=2$ we present a relevant example due to Gady Kozma, while the general statement for transient graphs is obtained by proving that for every transient irreducible Markov chain on a countable state space, which admits a stationary measure, its trace is a.s. recurrent for simple random walk. The case that the Markov chain is reversible is due to Gurel-Gurevich, Lyons and the first named author (2007). We exploit recent results in potential theory of non-reversible Markov chains in order to extend their result to the non-reversible setup.
The question of recurrence and transience of branching Markov chains is more subtle than for ordinary Markov chains; they can be classified in transience, weak recurrence, and strong recurrence. We review criteria for transience and weak recurrence and give several new conditions for weak recurrence and strong recurrence. These conditions make a unified treatment of known and new examples possible and provide enough information to distinguish between weak and strong recurrence. This represents a step towards a general classification of branching Markov chains. In particular, we show that in \e…
A simple model of the new notion of "Markov up" processes is proposed; its positive recurrence and ergodic properties are shown under the appropriate conditions.
We extend the Dirichlet principle to non-reversible Markov processes on countable state spaces. We present two variational formulas for the solution of the Poisson equation or, equivalently, for the capacity between two disjoint sets. As an application we prove a some recurrence theorems. In particular, we show the recurrence of two-dimensional cycle random walks under a second moment condition on the winding numbers.
In this paper, we investigate the properties of recurrent planar Markov random walks. More precisely, we study the set of recurrent points with the use of local limit theorems. The Nagaev-Guivarc'h spectral method provides several examples for which these local limit theorems are satisfied as soon as the (standard or non-standard) central limit theorem holds.
We develop criteria for recurrence and transience of one-dimensional Markov processes which have jumps and oscillate between $+\infty$ and $-\infty$. The conditions are based on a Markov chain which only consists of jumps (overshoots) of the process into complementary parts of the state space. In particular we show that a stable-like process with generator $-(-Δ)^{α(x)/2}$ such that $α(x)=α$ for $x<-R$ and $α(x)=β$ for $x>R$ for some $R>0$ and $α,β\in(0,2)$ is transient if and only if $α+β<2$, otherwise it is recurrent. As a special case this yields a new proof for the recurrence, point recurr…
We present a recurrence-transience classification for discrete-time Markov chains on manifolds with negative curvature. Our classification depends only on geometric quantities associated to the increments of the chain, defined via the Riemannian exponential map. We deduce that there exist Markov chains on a large class of such manifolds which are both recurrent and have zero average drift at every point. We give an explicit example of such a chain on hyperbolic space of arbitrary dimension, and also on a stochastically incomplete manifold. We also prove that such recurrent chains cannot be uni…
We characterize recurrence and transience of nonnegative multivariate autoregressive processes of order one with random contractive coefficient matrix, of subcritical multitype Galton-Watson branching processes in random environment with immigration, and of the related max-autoregressive processes and general random exchange processes. Our criterion is given in terms of the maximal Lyapunov exponent of the coefficient matrix and the cumulative distribution function of the innovation/immigration component.
A $ϕ$-irreducible and aperiodic Markov chain with stationary probability distribution will converge to its stationary distribution from almost all starting points. The property of Harris recurrence allows us to replace ``almost all'' by ``all,'' which is potentially important when running Markov chain Monte Carlo algorithms. Full-dimensional Metropolis--Hastings algorithms are known to be Harris recurrent. In this paper, we consider conditions under which Metropolis-within-Gibbs and trans-dimensional Markov chains are or are not Harris recurrent. We present a simple but natural two-dimensional…
We develop sharp large deviation asymptotics for the probability of ruin in a Markov-dependent stochastic economic environment and study the extremes for some related Markovian processes which arise in financial and insurance mathematics, related to perpetuities and the $\operatorname {ARCH}(1)$ and $\operatorname {GARCH}(1,1)$ time series models. Our results build upon work of Goldie [Ann. Appl. Probab. 1 (1991) 126--166], who has developed tail asymptotics applicable for independent sequences of random variables subject to a random recurrence equation. In contrast, we adopt a general approac…
We study Markov processes conditioned so that their local time must grow slower than a prescribed function. Building upon recent work on Brownian motion with constrained local time in [5] and [33], we study transience and recurrence for a broad class of Markov processes. In order to understand the distribution of the local time, we determine the distribution of a non-decreasing Lévy process (the inverse local time) conditioned to remain above a given level which varies in time. We study a time-dependent region, in contrast to previous works in which a process is conditioned to remain in a fixe…
We give three different criteria for transience of a Branching Markov Chain. These conditions enable us to give a classification of Branching Random Walks in Random Environment (BRWRE) on Cayley Graphs in recurrence and transience. This classification is stated explicitly for BRWRE on $\Z^d.$ Furthermore, we emphasize the interplay between Branching Markov Chains and the spectral radius. We prove properties of the spectral radius of the Random Walk in Random Environment with the help of appropriate Branching Markov Chains.
In this paper, we introduce an index which measures the strength of recurrence of symmetric Markov processes, and give some sufficient conditions for recurrence of direct products of symmetric diffusion processes. The index is given by the Dirichlet forms of the Markov processes. Moreover, as an application, we prove the recurrence of some multi-dimensional diffusion processes in random environments including zero potentials.
We study positive recurrence and transience of a two-station network in which the behavior of the server in each station is governed by a Markov chain with a finite number of server states; this service process can represent various service disciplines such as a non-preemptive priority service and K-limited service. Assuming that exogenous customers arrive according to independent Markovian arrival processes (MAPs), we represent the behavior of the whole network as a continuous-time Markov chain and, by the uniformization technique, obtain the corresponding discrete-time Markov chain, which is…
We show that the edges crossed by a random walk in a network form a recurrent graph a.s. In fact, the same is true when those edges are weighted by the number of crossings.
We give recurrence and transience criteria for two cases of time-homogeneous Markov chains on the real line with transition kernel $p(x,dy)=f_x(y-x)dy$, where $f_x(y)$ are probability densities of symmetric distributions and, for large $|y|$, have a power-law decay with exponent $α(x)+1$, with $α(x)\in(0,2)$. If $f_x(y)$ is the density of a symmetric $α$-stable distribution for negative $x$ and the density of a symmetric $β$-stable distribution for non-negative $x$, where $α,β\in(0,2)$, then the chain is recurrent if and only if $α+β\geq2.$ If the function $x\longmapsto f_x$ is periodic and if…
We consider moments of the return times (or first hitting times) in a discrete time discrete space Markov chain. It is classical that the finiteness of the first moment of a return time of one state implies the finiteness of the first moment of the first return time of any other state. We extend this statement to moments with respect to a function $f$, where $f$ satisfies a certain, best possible condition. This generalizes results of K. L. Chung (1954) who considered the functions $f(n)=n^p$ and wondered "[...] what property of the power $n^p$ lies behind this theorem [...]" (see Chung (1967)…
The recurrence features of persistent random walks built from variable length Markov chains are investigated. We observe that these stochastic processes can be seen as L{é}vy walks for which the persistence times depend on some internal Markov chain: they admit Markov random walk skeletons. A recurrence versus transience dichotomy is highlighted. We first give a sufficient Fourier criterion for the recurrence, close to the usual Chung-Fuchs one, assuming in addition the positive recurrence of the driving chain and a series criterion is derived. The key tool is the Nagaev-Guivarc'h method. Fina…
Using ideas borrowed from topological dynamics and ergodic theory we introduce topological and metric versions of the recurrence property for general Markov chains. The main question of interest here is how large is the set of recurrent points. We show that under some mild technical assumptions the set of non recurrent points is of zero reference measure. Necessary and sufficient conditions for a reference measure $m$ (which needs not to be dynamically invariant) to satisfy this property are obtained. These results are new even in the purely deterministic setting.
Positive recurrence of a $d$-dimensional diffusion with switching and with one recurrent and one transient regimes and variable switching intensities is established under suitable conditions. The approach is based on embedded Markov chains.
We present a coherent approach to recurrence and transience, starting from a version of the Riesz decomposition theorem for superharmonic elements. Our approach allows straightforward proofs of some known results, entails new theorems, and has applications to other aspects of completely positive operators: It leads to a classification of idempotent Markov operators, thereby identifying concretely the Choi-Effros product, which can be introduced on the range of these maps. We obtain an abstract Poisson integral and a representation theorem for idempotent entanglement breaking channels.
This paper addresses the question when the underlying Markov process of a multiclass queueing network is positive Harris recurrent. It is well-known that stability of the fluid limit model is a sufficient condition for this. Hence, stability of fluid (limit) models is of vital interest. Recently, it has been shown that if the fluid model satisfies certain properties, it is stable if and only if there exists a Lyapunov function. In this paper a new method is provided to conclude that the underlying Markov process is positive Harris recurrent if the fluid model is stable by using explicitly the …
We study the recurrence/transience phase transition for Markov chains on $\mathbb{R}_+$, $\mathbb{R}$, and $\mathbb{R}^2$ whose increments have heavy tails with exponent in $(1,2)$ and asymptotically zero mean. This is the infinite-variance analogue of the classical Lamperti problem. On $\mathbb{R}_+$, for example, we show that if the tail of the positive increments is about $c y^{-α}$ for an exponent $α\in (1,2)$ and if the drift at $x$ is about $b x^{-γ}$, then the critical regime has $γ= α-1$ and recurrence/transience is determined by the sign of $b + cπ\textrm{cosec} (πα)$. On $\mathbb{R}$…
A transition matrix $[U_{i,j}]_{i,j\geq 0}$ on $\mathbb{N}$ is said to be almost upper triangular if $U_{i,j}\geq 0\Rightarrow j\geq i-1$, so that the increments of the corresponding Markov chains are at least $-1$; a transition matrix $[L_{i,j}]_{i,j\geq 0}$ is said to be almost lower triangular if $L_{i,j}\geq 0\Rightarrow j\leq i+1$, and then, the increments of the corresponding Markov chains are at most $+1$. In the present paper, we characterize the recurrence, positive recurrence and invariant distribution for the class of almost triangular transition matrices. The upper case appears to …
In the present paper, we construct QMCs associated with Open Quantum Random Walks such that the transition operator of the chain is defined by OQRW and the restriction of QMC to the commutative subalgebra coincides with the distribution $P_ρ$ of OQRW. This sheds new light on some properties of the measure $P_ρ$. As an example, we simply mention that the measure can be considered as a distribution of some functions of certain Markov process. Furthermore, we study several properties of QMC and associated measure. A new notion of $\f$-recurrence of QMC is studied, and it is established relations …
We analyze a tractable model of a limit order book on short time scales, where the dynamics are driven by stochastic fluctuations between supply and demand. We establish the existence of a limiting distribution for the highest bid, and for the lowest ask, where the limiting distributions are confined between two thresholds. We make extensive use of fluid limits in order to establish recurrence properties of the model. We use the model to analyze various high-frequency trading strategies, and comment on the Nash equilibria that emerge between high-frequency traders when a market in continuous t…
This paper provides full classification of dynamics for continuous time Markov chains (CTMCs) on the non-negative integers with polynomial transition rate functions. Such stochastic processes are abundant in applications, in particular in biology. More precisely, for CTMCs of bounded jumps, we provide necessary and sufficient conditions in terms of calculable parameters for explosivity, recurrence vs transience, certain absorption, positive recurrence vs null recurrence, and implosivity. Simple sufficient conditions for exponential ergodicity of stationary distributions and quasi-stationary di…
We show that certain Markov jump processes associated to crystal growth models are positive recurrent when the parameters satisfy a rather natural condition.
We provide sufficient conditions for the validity of a dichotomy, i.e. zero-one law, between recurrence and transience of general frog models. In particular, the results cover frog models with i.i.d. numbers of frogs per site where the frog dynamics are given by quasi-transitive Markov chains or by random walks in a common random environment including super-critical percolation clusters on $\mathbb{Z}^d$. We also give a sufficient and almost sharp condition for recurrence of uniformly elliptic frog processes on $\mathbb{Z}^d$. Its proof uses the general zero-one law.
We consider the following problem in one-dimensional diffusion-limited aggregation (DLA). At time $t$, we have an "aggregate" consisting of $\Bbb{Z}\cap[0,R(t)]$ [with $R(t)$ a positive integer]. We also have $N(i,t)$ particles at $i$, $i>R(t)$. All these particles perform independent continuous-time symmetric simple random walks until the first time $t'>t$ at which some particle tries to jump from $R(t)+1$ to $R(t)$. The aggregate is then increased to the integers in $[0,R(t')]=[0,R(t)+1]$ [so that $R(t')=R(t)+1$] and all particles which were at $R(t)+1$ at time $t'{-}$ are removed from the s…
We consider a discrete time population model for which each individual alive at time $n$ survives independently of everybody else at time $n+1$ with probability $β_n$. The sequence $(β_n)$ is i.i.d. and constitutes our random environment. Moreover, at every time $n$ we add $Z_n$ individuals to the population. The sequence $(Z_n)$ is also i.i.d. We find sufficient conditions for null recurrence and transience (positive recurrence has been addressed by Neuts). We apply our results to a particular $(Z_n)$ distribution and deterministic $β$. This particular case shows a rather unusual phase transi…
The Tree Builder Random Walk is a special random walk that evolves on trees whose size increases with time, randomly and depending upon the walker. After every s steps of the walker, a random number of vertices are added to the tree and attached to the current position of the walker. These processes share similarities with other important classes of markovian and non-markovian random walks presenting a large variety of behaviors according to parameters specifications. We show that for a large and most significant class of tree builder random walks, the process is either null recurrent or trans…
A stable-like Markov chain is a time-homogeneous Markov chain on the real line with the transition kernel $p(x,dy)=f_x(y-x)dy$, where the density functions $f_x(y)$, for large $|y|$, have a power-law decay with exponent $α(x)+1$, where $α(x)\in(0,2)$. In this paper, under a certain uniformity condition on the density functions $f_x(y)$ and additional mild drift conditions, we give sufficient conditions for recurrence in the case when $0<\liminf_{|x|\longrightarrow\infty}α(x)$, sufficient conditions for transience in the case when $\limsup_{|x|\longrightarrow\infty}α(x)<2$ and sufficient condit…
For i.i.d. random vectors $(M_{1},Q_{1}),(M_{2},Q_{2}),\ldots$ such that $M>0$ a.s., $Q\geq 0$ a.s. and $\mathbb{P}(Q=0)<1$, the random difference equation $X_{n}=M_{n}X_{n-1}+Q_{n}$, $n=1,2,\ldots$, is studied in the critical case when the random walk with increments $\log M_{1},\log M_{2}$ is oscillating. We provide conditions for the null-recurrence and transience of the Markov chain $(X_{n})_{n\ge 0}$ by inter alia drawing on techniques developed in the related article Alsmeyer et al (2017) for another case exhibiting the null-recurrence/transience dichotomy.
We give a short proof of Theorem 2.1 from [MR07], stating that the linearly edge reinforced random walk (ERRW) on a locally finite graph is recurrent if and only if it returns to its starting point almost surely. This result was proved in [MR07] by means of the much stronger property that the law of the ERRW is a mixture of Markov chains. Our proof only uses this latter property on finite graphs, in which case it is a consequence of De Finetti's theorem on exchangeability.
We consider the recurrence and transience problem for a time-homogeneous Markov chain on the real line with transition kernel $p(x,\mathrm{d}y)=f_x(y-x)\,\mathrm{d}y$, where the density functions $f_x(y)$, for large $|y|$, have a power-law decay with exponent $α(x)+1$, where $α(x)\in(0,2)$. In this paper, under a uniformity condition on the density functions $f_x(y)$ and an additional mild drift condition, we prove that when $\lim\inf_{|x|\longrightarrow\infty}α(x)>1$, the chain is recurrent. Similarly, under the same uniformity condition on the density functions $f_x(y)$ and some mild technic…
One dimensional excited random walk has been extensively studied for bounded, i.i.d. cookie environments. In this case, many important properties of the walk including transience or recurrence, positivity or non-positivity of the speed, and the limiting distribution of the position of the walker are all characterized by a single parameter $δ$, the total expected drift per site. In the more general case of stationary ergodic environments, things are not so well understood. If all cookies are positive then the same threshold for transience vs. recurrence holds, even if the cookie stacks are unbo…
Catastrophe Markov chain population models have received a lot of attention in the recent past. We herewith consider two special cases of such models involving total disasters, both in discrete and in continuous-time. Depending on the parameters range, the two models can show up a recurrence/transience transition and, in the critical case, a positive/null recurrence transition. The collapse transition probabilities are chosen in such a way that the models are exactly solvable and, in case of positive recurrence, intimately related to the extended Sibuya and Pareto-Zipf distributions whose divi…
All one-condition generalized inverses of the Markovian kernel I - P, where P is the transition matrix of a finite irreducible Markov chain, can be uniquely specified in terms of the stationary probabilities and the mean first passage times of the underlying Markov chain. Special sub-families include the group inverse of I - P, Kemeny and Snell's fundamental matrix of the Markov chain and the Moore-Penrose g-inverse. The elements of some sub-families of the generalized inverses can also be re-expressed involving the second moments of the recurrence time variables. Some applications to Kemeny's…
This paper is devoted to the study of continuous-time processes known as continuous-time open quantum walks (CTOQWs). A CTOQW represents the evolution of a quantum particle constrained to move on a discrete graph, but also has internal degrees of freedom modeled by a state (in the quantum mechanical sense), and contain as a special case continuous-time Markov chains on graphs. Recurrence and transience of a vertex are an important notion in the study of Markov chains, and it is known that all vertices must be of the same nature if the Markov chain is irreducible. In the present paper we addres…
Given a sequence $(M_{k}, Q_{k})_{k\ge 1}$ of independent, identically distributed ran\-dom vectors with nonnegative components, we consider the recursive Markov chain $(X_{n})_{n\ge 0}$, defined by the random difference equation $X_{n}=M_{n}X_{n-1}+Q_{n}$ for $n\ge 1$, where $X_{0}$ is independent of $(M_{k}, Q_{k})_{k\ge 1}$. Criteria for the null-recurrence/transience are provided in the situation where $(X_{n})_{n\ge 0}$ is contractive in the sense that $M_{1}\cdot\ldots\cdot M_{n}\to 0$ a.s., yet occasional large values of the $Q_{n}$ overcompensate the contractive behavior so that positi…
Predictive constructions are a powerful way of characterizing the probability law of stochastic processes with certain forms of invariance, such as exchangeability or Markov exchangeability. When de Finetti-like representation theorems are available, the predictive characterization implicitly defines the prior distribution, starting from assumptions on the observables; moreover, it often helps designing efficient computational strategies. In this paper we give necessary and sufficient conditions on the sequence of predictive distributions such that they characterize a Markov exchangeable proba…
This paper contributes an in-depth study of properties of continuous time Markov chains (CTMCs) on non-negative integer lattices $\N_0^d$, with particular interest in one-dimensional CTMCs with polynomial transitions rates. Such stochastic processes are abundant in applications, in particular in biology. We characterize the structure of the state space of general CTMCs on $\N_0^d$ in terms of the set of jump vectors and their corresponding transition rate functions. For CTMCs on $\N_0$ with polynomial transition rate functions, we provide threshold criteria in terms of easily computable parame…
Time change is one of the most basic and very useful transformations for Markov processes. The time changed process can also be regarded as the trace of the original process on the support of the Revuz measure used in the time change. In this paper we give a complete characterization of time changed processes of an arbitrary symmetric Markov process, in terms of the Beurling--Deny decomposition of their associated Dirichlet forms and of Feller measures of the process. In particular, we determine the jumping and killing measure (or, equivalently, the Lévy system) for the time-changed process. W…
Famously, a $d$-dimensional, spatially homogeneous random walk whose increments are non-degenerate, have finite second moments, and have zero mean is recurrent if $d \in \{1,2\}$ but transient if $d \geq 3$. Once spatial homogeneity is relaxed, this is no longer true. We study a family of zero-drift spatially non-homogeneous random walks (Markov processes) whose increment covariance matrix is asymptotically constant along rays from the origin, and which, in any ambient dimension $d \geq 2$, can be adjusted so that the walk is either transient or recurrent. Natural examples are provided by rand…
We study a particular class of trace-preserving completely positive maps, called PQ-channels, for which classical and quantum evolutions are isolated in a certain sense. By combining open quantum random walks with a notion of recurrence, we are able to describe criteria for recurrence of the walk related to this class of channels. Positive recurrence for open walks is also discussed in this context.
In a finite state irreducible Markov chain with stationary probabilities π_i and mean first passage times m_(ij) (mean recurrence time when i = j) it was first shown by Kemeny and Snell (1960) that \sum_j π_j m_(ij) is a constant K, not depending on i. This constant has since become known as Kemeny's constant. A variety of techniques for finding expressions and various bounds for K are derived. The main interpretation focuses on its role as the expected time to mixing in a Markov chain. Various applications are considered including perturbation results, mixing on directed graphs and its relati…
Simple random walks are a basic staple of the foundation of probability theory and form the building block of many useful and complex stochastic processes. In this paper we study a natural generalization of the random walk to a process in which the allowed step sizes take values in the set $\{\pm1,\pm2,\ldots,\pm k\}$, a process we call a random leap. The need to analyze such models arises naturally in modern-day data science and so-called "big data" applications. We provide closed-form expressions for quantities associated with first passage times and absorption events of random leaps. These …
We study the asymptotic behaviour of Markov chains $(X_n,η_n)$ on $\mathbb{Z}_+ \times S$, where $\mathbb{Z}_+$ is the non-negative integers and $S$ is a finite set. Neither coordinate is assumed to be Markov. We assume a moments bound on the jumps of $X_n$, and that, roughly speaking, $η_n$ is close to being Markov when $X_n$ is large. This departure from much of the literature, which assumes that $η_n$ is itself a Markov chain, enables us to probe precisely the recurrence phase transitions by assuming asymptotically zero drift for $X_n$ given $η_n$. We give a recurrence classification in ter…
We study a Markov chain on $\mathbb{R}_+ \times S$, where $\mathbb{R}_+$ is the non-negative real numbers and $S$ is a finite set, in which when the $\mathbb{R}_+$-coordinate is large, the $S$-coordinate of the process is approximately Markov with stationary distribution $π_i$ on $S$. If $μ_i(x)$ is the mean drift of the $\mathbb{R}_+$-coordinate of the process at $(x,i) \in \mathbb{R}_+ \times S$, we study the case where $\sum_{i} π_i μ_i (x) \to 0$, which is the critical regime for the recurrence-transience phase transition. If $μ_i(x) \to 0$ for all $i$, it is natural to study the Lamperti …
We consider a discrete-time Markov chain $(X^t,Y^t)$, $t=0,1,2,...$, where the $X$-component forms a Markov chain itself. Assume that $(X^t)$ is Harris-ergodic and consider an auxiliary Markov chain ${\hat{Y}^t}$ whose transition probabilities are the averages of transition probabilities of the $Y$-component of the $(X,Y)$-chain, where the averaging is weighted by the stationary distribution of the $X$-component. We first provide natural conditions in terms of test functions ensuring that the $\hat{Y}$-chain is positive recurrent and then prove that these conditions are also sufficient for pos…
We derive some key extremal features for $k$th order Markov chains that can be used to understand how the process moves between an extreme state and the body of the process. The chains are studied given that there is an exceedance of a threshold, as the threshold tends to the upper endpoint of the distribution. Unlike previous studies with $k>1$, we consider processes where standard limit theory describes each extreme event as a single observation without any information about the transition to and from the body of the distribution. Our work uses different asymptotic theory which results in no…
