In the present paper we show that for any given digraph $\mathbb{G} =([n], \vec{E})$, i.e. an oriented graph without self-loops and 2-cycles, one can construct a 1-dependent Markov chain and $n$ identically distributed hitting times $T_1, \ldots , T_n $ on this chain such that the probability of the event $T_i > T_j $, for any $i, j = 1, \ldots n$, is larger than $\frac{1}{2}$ if and only if $(i,j)\in \vec{E}$. This result is related to various paradoxes in probability theory, concerning in particular non-transitive dice.
We consider the problem of characterising expected hitting times and hitting probabilities for imprecise Markov chains. To this end, we consider three distinct ways in which imprecise Markov chains have been defined in the literature: as sets of homogeneous Markov chains, as sets of more general stochastic processes, and as game-theoretic probability models. Our first contribution is that all these different types of imprecise Markov chains have the same lower and upper expected hitting times, and similarly the hitting probabilities are the same for these three types. Moreover, we provide a ch…
In this paper we are concerned with hitting times of a family of density-dependent Markov chains. A moderate deviation principle of the hitting time is given. The proof of the main theorem relies heavily on moderate deviations of density-dependent Markov chains given in \cite{Xue2021} and upper bounds of large deviations of Markov processes given in \cite{Dupuis1991}. An analogue moderate deviation of the hitting time of the diffusion approximation of the density-dependent Markov chain introduced in \cite{Ethier1986} is also given.
The paper presents new asymptotic recurrent algorithms of phase space reduction for regularly and singularly perturbed semi-Markov processes. These algorithms give effective conditions of weak convergence for distributions and convergence of expectations for hitting times as well as recurrent formulas for computing the corresponding normalisation functions, Laplace transforms for limiting distributions and limits for expectations.
Extending the approach of the paper [Mathieu, P. (1997) Hitting times and spectral gap inequalities, Ann. Inst. Henri Poincare 33, 4, 437 -- 465], we prove that the Poincare inequality for a (possibly non-symmetric) Markov process yields the exponential integrability of the hitting times of this process. For symmetric elliptic diffusions, this provides a criterion for the Poincare inequality in the terms of hitting times.
The hitting and mixing times are two fundamental quantities associated with Markov chains. In Peres and Sousi[PS2015] and Oliveira[Oli2012], the authors show that the mixing times and "worst-case" hitting times of reversible Markov chains on finite state spaces are equal up to some universal multiplicative constant. We use tools from nonstandard analysis to extend this result to reversible Markov chains on general state spaces that satisfy the strong Feller property. Finally, we show that this asymptotic equivalence can be used to find bounds on the mixing times of a large class of Markov chai…
Consider a discrete time, ergodic Markov chain with finite state space which is started from stationarity. Fill and Lyzinski (2014) showed that, in some cases, the hitting time for a given state may be represented as a sum of a geometric number of IID random variables. We extend this result by giving explicit bounds on the distance between any such hitting time and an appropriately chosen geometric sum, along with other related approximations. The compounding random variable in our approximating geometric sum is a strong stationary time for the underlying Markov chain; we also discuss the appr…
We develop some sufficient conditions for the stochastic ordering between hitting times, in a fixed state, for two Markov chains. In particular, we focus attention on the so called \emph{skip-free} case. In the analysis of such a case, we develop a special type of coupling. We also compare different types of relations between two, non-necessarily skip-free, Markov chains on the same state space. Such relations have a natural role in establishing the usual and the asymptotic stochastic ordering between the probability distributions of hitting times. Finally, we present some discussions and exam…
A well-known theorem for an irreducible skip-free chain with absorbing state $d$, under some conditions, is that the hitting (absorbing) time of state $d$ starting from state 0 is distributed as the sum of $d$ independent geometric (or exponential) random variables. The purpose of this paper is to present a direct and simple proof of the theorem in the cases of both discrete and continuous time skip-free Markov chains. Our proof is to calculate directly the generation functions (or Laplace transforms) of hitting times in terms of the iteration method.
We study the problem of characterizing the expected hitting times for a robust generalization of continuous-time Markov chains. This generalization is based on the theory of imprecise probabilities, and the models with which we work essentially constitute sets of stochastic processes. Their inferences are tight lower- and upper bounds with respect to variation within these sets. We consider three distinct types of these models, corresponding to different levels of generality and structural independence assumptions on the constituent processes. Our main results are twofold; first, we demonstrat…
Motivated by a risk process with positive and negative premium rates, we consider a real-valued Markov additive process with finitely many background states. This additive process linearly increases or decreases while the background state is unchanged, and may have upward jumps at the transition instants of the background state. It is known that the hitting probabilities of this additive process at lower levels have a matrix exponential form. We here study the hitting probabilities at upper levels, which do not have a matrix exponential form in general. These probabilities give the ruin probab…
Let 0<α<1/2. We show that the mixing time of a continuous-time reversible Markov chain on a finite state space is about as large as the largest expected hitting time of a subset of stationary measure at least αof the state space. Suitably modified results hold in discrete time and/or without the reversibility assumption. The key technical tool is a construction of a random set A such that the hitting time of A is both light-tailed and a stationary time for the chain. We note that essentially the same results were obtained independently by Peres and Sousi [arXiv:1108.0133].
We develop a systematic matrix-analytic approach, based on intertwinings of Markov semigroups, for proving theorems about hitting-time distributions for finite-state Markov chains -- an approach that (sometimes) deepens understanding of the theorems by providing corresponding sample-path-by-sample-path stochastic constructions. We employ our approach to give new proofs and constructions for two theorems due to Mark Brown, theorems giving two quite different representations of hitting-time distributions for finite-state Markov chains started in stationarity. The proof, and corresponding constru…
We present a novel algorithm to solve a non-linear system of equations, whose solution can be interpreted as a tight lower bound on the vector of expected hitting times of a Markov chain whose transition probabilities are only partially specified. We also briefly sketch how this method can be modified to solve a conjugate system of equations that gives rise to the corresponding upper bound. We prove the correctness of our method, and show that it converges to the correct solution in a finite number of steps under mild conditions on the system. We compare the runtime complexity of our method to…
For a finite state Markov process and a finite collection $\{ Γ_k, k \in K \}$ of subsets of its state space, let $τ_k$ be the first time the process visits the set $Γ_k$. We derive explicit/recursive formulas for the joint density and tail probabilities of the stopping times $\{ τ_k, k \in K\}$. The formulas are natural generalizations of those associated with the jump times of a simple Poisson process. We give a numerical example and indicate the relevance of our results to credit risk modeling.
For any discrete target distribution, we exploit the connection between Markov chains and Stein's method via the generator approach and express the solution of Stein's equation in terms of expected hitting time. This yields new upper bounds of Stein's factors in terms of the parameters of the Markov chain, such as mixing time and the gradient of expected hitting time. We compare the performance of these bounds with those in the literature, and in particular we consider Stein's method for discrete uniform, binomial, geometric and hypergeometric distribution. As another application, the same met…
We consider finite irreducible Markov chains. It was shown that mean hitting time from one state to another satisfies the triangle inequality. Hence, sum of mean hitting time between couple of states in both directions is a metric on the space of states.
We generalize the notion of strong stationary time and we give a representation formula for the hitting time to a target set in the general case of non-reversible Markov processes.
New algorithms for computing power moments of hitting times and accumulated rewards of hitting type for semi-Markov processes. The algorithms are based on special techniques of sequential phase space reduction and recurrence relations connecting moments of rewards. Applications are discussed as well as possible generalisations of presented results and results of experimental studies.
We consider a continuous-time Markov chain with a finite or countable state space. For a site y and subset H of the state space, the hitting time of y under taboo H is defined to be infinite if the process trajectory hits H before y, and the first hitting time of y otherwise. We investigate the probability that such times are finite. In particular, if the taboo set is finite, an efficient iterative scheme reduces the study to the known case of a singleton taboo. A similar procedure applies in the case of finite complement of the taboo set. The study is motivated by classification of branching …
Generally the convergence rate in exponential ergodicity $λ$ is an upper bound for the convergence rate $κ$ in uniform ergodicity for a Markov process, that is $λ\geqslantκ$. In this paper, we prove that $κ\geqslant \inf \{lambda,1/M_H\}$, where $M_H$ is a uniform bound on the moment of the hitting time to a "compact" set $H$. In the case where $M_H$ can be made arbitrarily small for $H$ large enough, we obtain that $λ=κ$. The general results are applied to Markov chains, diffusion processes and solutions to SDEs driven by symmetric stable processes.
In the absence of acceleration, the velocity formula gives "distance travelled equals speed multiplied by time". For a broad class of Markov chains such as circulant Markov chains or random walk on complete graphs, we prove a probabilistic analogue of the velocity formula between entropy and hitting time, where distance is the entropy of the Markov trajectories from state $i$ to state $j$ in the sense of [L. Ekroot and T. M. Cover. The entropy of Markov trajectories. IEEE Trans. Inform. Theory 39(4): 1418-1421.], speed is the classical entropy rate of the chain, and the time variable is the ex…
Given a discrete source distribution $μ$ and discrete target distribution $ν$ on a common finite state space $\mathcal{X}$, we are tasked with transporting $μ$ to $ν$ using a given discrete-time Markov chain $X$ with the quickest possible time on average. We define the optimal transport time $H(μ,ν)$ as stopping rule of $X$ that gives the minimial expected transport time. This is also known as the access time from $μ$ to $ν$ of $X$ in [L. Lovász and P. Winkler. Efficient Stopping Rules for Markov Chains. Proceedings of the Twenty-seventh Annual ACM Symposium on Theory of Computing (STOC '95) 7…
For any two-sided jumping $α$-stable process, where $1 < α< 2$, we find an explicit identity for the law of the first hitting time of the origin. This complements existing work in the symmetric case and the spectrally one-sided case; cf. Yano-Yano-Yor (2009) and Cordero (2010), and Peskir (2008) respectively. We appeal to the Lamperti-Kiu representation of Chaumont-Pantí-Rivero (2011) for real-valued self-similar Markov processes. Our main result follows by considering a vector-valued functional equation for the Mellin transform of the integrated exponential Markov additive process in the Lamp…
Given an irreducible discrete-time Markov chain on a finite state space, we consider the largest expected hitting time $T(α)$ of a set of stationary measure at least $α$ for $α\in(0,1)$. We obtain tight inequalities among the values of $T(α)$ for different choices of $α$. One consequence is that $T(α) \le T(1/2)/α$ for all $α< 1/2$. As a corollary we have that, if the chain is lazy in a certain sense as well as reversible, then $T(1/2)$ is equivalent to the chain's mixing time, answering a question of Peres. We furthermore demonstrate that the inequalities we establish give an almost everywher…
There are several works characterizing the total-variation mixing time of a reversible Markov chain in term of natural probabilistic concepts such as stopping times and hitting times. In contrast, there is no known analog for the $L_{2}$ mixing time, $τ_{2}$ (while there are sophisticated analytic tools to bound $ τ_2$, in general they do not determine $τ_2$ up to a constant factor and they lack a probabilistic interpretation). In this work we show that $τ_2$ can be characterized up to a constant factor using hitting times distributions. We also derive a new extremal characterization of the Lo…
Motivated by the notion of resistance distance on graph, we define a new resistance distance between two states on a given finite ergodic Markov chain based on its fundamental matrix. We prove a few equivalent formulations and discuss its relation with other parameters of the Markov chain such as its group inverse, stationary distribution, eigenvalues or hitting time. In addition, building upon existing sum rules for the hitting time of Markov chain, we give sum rules of this new resistance distance of Markov chains that resembles the sum rules of the resistance distance on graph. This yields …
We study a class of self-similar jump type SDEs driven by Hölder-continuous drift and noise coefficients. Using the Lamperti transformation for positive self-similar Markov processes we obtain a necessary and sufficient condition for almost sure extinction in finite time. We then show that for certain parameters pathwise uniqueness holds in a restricted sense, namely among solutions spending a Lebesgue-negligible amount of time at 0. A direct power transformation plays a key role.
We study the two-dimensional joint distribution of the first hitting time of a constant level by a continuous-state branching process with immigration and their primitive stopped at this time. We show an explicit expression of its Laplace transform. Using this formula, we study the polarity of zero and provide a necessary and sufficient criterion for transience or recurrence. We follow the approach of Shiga, T. (1990) [A recurrence criterion for Markov processes of Ornstein-Uhlenbeck type. Probability Theory and Related Fields, 85(4), 425-447], by finding some $λ$-invariant functions for the g…
Let $X$ be a regular linear continuous positively recurrent Markov process with state space $\R$, scale function $S$ and speed measure $m$. For $a\in \R$ denote B^+_a&=\sup_{x\geq a} \m(]x,+\infty[)(S(x)-S(a)) B^-_a&=\sup_{x\leq a} \m(]-\infty;x[)(S(a)-S(x)) We study some characteristic relations between $B^+_a$, $B^-_a$, the exponential moments of the hitting times $T_a$ of $X$, the Hardy and Poincaré inequalities for the Dirichlet form associated with $X$. As a corollary, we establish the equivalence between the existence of exponential moments of the hitting times and the spectral gap of th…
The problem of missing mass in statistical inference (posed by McAllester and Ortiz, NIPS'02; most recently revisited by Changa and Thangaraj, ISIT'2019) seeks to estimate the weight of symbols that have not been sampled yet from a source. So far all the approaches have been focused on the IID model which, although overly simplistic, is already not straightforward to tackle. The non-trivial part is in handling correlated events and sums of variables with very different scales where classical concentration inequalities do not yield good bounds. In this paper we develop the research on missing m…
We present an algorithm that can efficiently compute a broad class of inferences for discrete-time imprecise Markov chains, a generalised type of Markov chains that allows one to take into account partially specified probabilities and other types of model uncertainty. The class of inferences that we consider contains, as special cases, tight lower and upper bounds on expected hitting times, on hitting probabilities and on expectations of functions that are a sum or product of simpler ones. Our algorithm exploits the specific structure that is inherent in all these inferences: they admit a gene…
Consider a sequence of continuous-time irreducible reversible Markov chains and a sequence of initial distributions, $μ_n$. The sequence is said to exhibit $μ_n$-cutoff if the convergence to stationarity in total variation distance is abrupt, w.r.t. this sequence of initial distributions. In this work we give a characterization of $μ_n$-cutoff for an arbitrary sequence of initial distributions $μ_n$ (in the above setup). Our characterization is expressed in terms of hitting times of sets which are "worst" w.r.t. $μ_n$. Consider a Markov chain on $Ω$ whose stationary distribution in $π$. Let $t…
In this article, we obtain properties of the law associated to the first hitting time of a threshold by a one-dimensional uniformly elliptic diffusion process and to the associated process stopped at the threshold. Our methodology relies on the parametrix method that we apply to the associated Markov semigroup. It allows to obtain explicit expressions for the corresponding transition densities and to study its regularity properties up to the boundary under mild assumptions on the coefficients. As a by product, we also provide Gaussian upper estimates for these laws and derive a probabilistic r…
In this paper, we show how to use stochastic approximation to compute hitting time of a stochastic process, based on the study of the time for a fluid approximation of this process to be at distance 1/N of its fixed point. This approach is developed to study a generalized version of the coupon collector problem. The system is composed by N independent identical Markov chains. At each time step, one Markov chain is picked at random and performs one transition. We show that the time at which all chains have hit the same state is bounded by a N log N + b N log log N + O(N) where a and b are two c…
In a recent paper in the Journal of Theoretical Probability Gong, Mao and Zhang, using the theory of Dirichlet forms, extended Karlin and McGregor's classical results on first-hitting times of a birth-death process on the nonnegative integers by establishing a representation for the Laplace transform $\mathbb{E}[e^{sT_{ij}}]$ of the first-hitting time $T_{ij}$ for $any$ pair of states $i$ and $j$, as well as asymptotics for $\mathbb{E}[e^{sT_{ij}}]$ when either $i$ or $j$ tends to infinity. It will be shown here that these results may also be obtained by employing tools from the orthogonal-pol…
Consider a negatively drifted one dimensional Brownian motion starting at positive initial position, its first hitting time to 0 has the inverse Gaussian law. Moreover, conditionally on this hitting time, the Brownian motion up to that time has the law of a 3- dimensional Bessel bridge. In this paper, we give a generalization of this result to a family of Brownian motions with interacting drifts, indexed by the vertices of a conductance network. The hitting times are equal in law to the inverse of a random potential that appears in the analysis of a self-interacting process called the Vertex R…
The "drift-and-minorization" method, introduced and popularized in (Rosenthal, 1995; Meyn and Tweedie, 1994; Meyn and Tweedie, 2012), remains the most popular approach for bounding the convergence rates of Markov chains used in statistical computation. This approach requires estimates of two quantities: the rate at which a single copy of the Markov chain "drifts" towards a fixed "small set", and a "minorization condition" which gives the worst-case time for two Markov chains started within the small set to couple with moderately large probability. In this paper, we build on (Oliveira, 2012; Pe…
We study a classical model for the accumulation of errors in multi-qubit quantum computations. By modeling the error process in a quantum computation using two coupled Markov chains, we are able to capture a weak form of time-dependency between errors in the past and future. By subsequently using techniques from the field of discrete probability theory, we calculate the probability that error quantities such as the fidelity and trace distance exceed a threshold analytically. The formulae cover fairly generic error distributions, cover multi-qubit scenarios, and are applicable to e.g. the rando…
We provide conditions that classify cover times for sequences of random walks on random graphs into two types: One type (Type 1) is the class of cover times that are of the order of the maximal hitting times scaled by the logarithm of the size of vertex sets. The other type (Type 2) is the class of cover times that are of the order of the maximal hitting times. The conditions are described by some parameters determined by the underlying graphs: the volumes, the diameters with respect to the resistance metric, the coverings or packings by balls in the resistance metric. We apply the conditions …
Consider the high-order heat-type equation $\partial u/\partial t=\pm\partial^N u/\partial x^N$ for an integer $N>2$ and introduce the related Markov pseudo-process $(X(t))_{t\ge 0}$. In this paper, we study several functionals related to $(X(t))_{t\ge 0}$: the maximum $M(t)$ and minimum $m(t)$ up to time $t$; the hitting times $τ_a^+$ and $τ_a^-$ of the half lines $(a,+\infty)$ and $(-\infty,a)$ respectively. We provide explicit expressions for the distributions of the vectors $(X(t),M(t))$ and $(X(t),m(t))$, as well as those of the vectors $(τ_a^+,X(τ_a^+))$ and $(τ_a^-,X(τ_a^-))$.
For a skip-free Markov process on non-negative integers with generator matrix Q, we evaluate the joint Laplace transform of the occupation times before hitting the state n (starting at 0). This Laplace transform has a very straightforward and familiar expression. We investigate the properties of this Laplace transform, especially the conditions under which the occupation times form a Markov chain.
In this Note we study optimal stopping problems for strong Markov processes and affine functions. We give a justification of the Snell envelope form using standard results of optimal stopping. We also justify the convexity of the value function, and without a priori restriction to a particular class of stopping times, we deduce that the smallest optimal stopping time is necessarily a hitting time.
In this paper, we consider the N-urn Ehrenfest model. By utilizing an auxiliary continuous-time Markov chain, we obtain the explicit formula for the Laplace transform of the hitting time from a single state to a set A of states where A satisfies some symmetric properties. After obtaining the Laplace transform, we are able to compute the high-order moments(especially, variance) for the hitting time.
We consider irreducible reversible discrete time Markov chains on a finite state space. Mixing times and hitting times are fundamental parameters of the chain. We relate them by showing that the mixing time of the lazy chain is equivalent to the maximum over initial states x and large sets A of the hitting time of A starting from x. We also prove that the first time when averaging over two consecutive time steps is close to stationarity is equivalent to the mixing time of the lazy version of the chain.
This paper studies limit theorems for Markov Chains with general state space under conditions which imply subgeometric ergodicity. We obtain a central limit theorem and moderate deviation principles for additive not necessarily bounded functional of the Markov chains under drift and minorization conditions which are weaker than the Foster-Lyapunov conditions. The regeneration-split chain method and a precise control of the modulated moment of the hitting time to small sets are employed in the proof.
We consider irreducible Markov chains on a finite state space. We show that the mixing time of any such chain is equivalent to the maximum, over initial states $x$ and moving large sets $(A_s)_s$, of the hitting time of $(A_s)_s$ starting from $x$. We prove that in the case of the $d$-dimensional torus the maximum hitting time of moving targets is equal to the maximum hitting time of stationary targets. Nevertheless, we construct a transitive graph where these two quantities are not equal, resolving an open question of Aldous and Fill on a "cat and mouse" game.
It is well known that the distributions of hitting times in Markov chains are quite irregular, unless the limit as time tends to infinity is considered. We show that nevertheless for a typical finite irreducible Markov chain and for nondegenerate initial distributions the tails of the distributions of the hitting times for the states of a Markov chain can be ordered, i.e., they do not overlap after a certain finite moment of time. If one considers instead each state of a Markov chain as a source rather than a sink then again the states can generically be ordered according to their efficiency. …
In the last years, many authors studied a class of continuous time semi-Markov processes obtained by time-changing Markov processes by hitting times of independent subordinators. Such processes are governed by integro-differential convolution equations of generalized fractional type. The aim of this paper is to develop the discrete-time version of such a theory. We show that a class of discrete-time semi-Markov chains can be seen as time-changed Markov chains and we obtain governing convolution type equations. Such processes converge weakly to those in continuous time under suitable scaling li…
We present results relating mixing times to the intersection time of branching random walk (BRW) in which the logarithm of the expected number of particles grows at rate of the spectral-gap $\mathrm{gap}$ . This is a finite state space analog of a critical branching process. Namely, we show that the maximal expected hitting time of a state by such a BRW is up to a universal constant larger than the $L_{\infty}$ mixing-time, whereas under transitivity the same is true for the intersection time of two independent such BRWs. Using the same methodology, we show that for a sequence of reversible Ma…
The rotor walk is a derandomized version of the random walk on a graph. On successive visits to any given vertex, the walker is routed to each of the neighboring vertices in some fixed cyclic order, rather than to a random sequence of neighbors. The concept generalizes naturally to Markov chains on a countable state space. Subject to general conditions, we prove that many natural quantities associated with the rotor walk (including normalized hitting frequencies, hitting times and occupation frequencies) concentrate around their expected values for the random walk. Furthermore, the concentrati…
