We investigate the local times of a continuous-time Markov chain on an arbitrary discrete state space. For fixed finite range of the Markov chain, we derive an explicit formula for the joint density of all local times on the range, at any fixed time. We use standard tools from the theory of stochastic processes and finite-dimensional complex calculus. We apply this formula in the following directions: (1) we derive large deviation upper estimates for the normalized local times beyond the exponential scale, (2) we derive the upper bound in Varadhan's \chwk{l}emma for any measurable functional of the local times, \ch{and} (3) we derive large deviation upper bounds for continuous-time simple random walk on large subboxes of $\Z^d$ tending to $\Z^d$ as time diverges. We finally discuss the relation of our density formula to the Ray-Knight theorem for continuous-time simple random walk on $\Z$, which is analogous to the well-known Ray-Knight description of Brownian local times. In this extended version, we prove that the Ray-Knight theorem follows from our density formula.
For a skip-free Markov process on non-negative integers with generator matrix Q, we evaluate the joint Laplace transform of the occupation times before hitting the state n (starting at 0). This Laplace transform has a very straightforward and familiar expression. We investigate the properties of this Laplace transform, especially the conditions under which the occupation times form a Markov chain.
For a finite state Markov process and a finite collection $\{ Γ_k, k \in K \}$ of subsets of its state space, let $τ_k$ be the first time the process visits the set $Γ_k$. We derive explicit/recursive formulas for the joint density and tail probabilities of the stopping times $\{ τ_k, k \in K\}$. The formulas are natural generalizations of those associated with the jump times of a simple Poisson process. We give a numerical example and indicate the relevance of our results to credit risk modeling.
This paper considers a work-conserving FIFO single-server queue with multiple batch Markovian arrival streams governed by a continuous-time finite-state Markov chain. A particular feature of this queue is that service time distributions of customers may be different for different arrival streams. After briefly discussing the actual waiting time distributions of customers from respective arrival streams, we derive a formula for the vector generating function of the time-average joint queue length distribution in terms of the virtual waiting time distribution. Further assuming the discrete phase…
We justify and discuss expressions for joint lower and upper expectations in imprecise probability trees, in terms of the sub- and supermartingales that can be associated with such trees. These imprecise probability trees can be seen as discrete-time stochastic processes with finite state sets and transition probabilities that are imprecise, in the sense that they are only known to belong to some convex closed set of probability measures. We derive various properties for their joint lower and upper expectations, and in particular a law of iterated expectations. We then focus on the special cas…
We consider the down/up crossing property of weighted Markov branching processes. The joint probability distribution of multi crossing numbers of such processes are obtained. In particular, for Markov branching processes, the probability distribution of death number is given and for bulk-arrival queueing model, the joint probability distributions of service number and arrival number is also given.
We study ergodic properties of a class of Markov-modulated general birth-death processes under fast regime switching. The first set of results concerns the ergodic properties of the properly scaled joint Markov process with a parameter that is taken large. Under very weak hypotheses, we show that if the averaged process is exponentially ergodic for large values of the parameter, then the same applies to the original joint Markov process. The second set of results concerns steady-state diffusion approximations, under the assumption that the 'averaged' fluid limit exists. Here, we establish conv…
In the paper we propose certain conditions, relatively easy to verify, which ensure the central limit theorem for some general class of Markov chains. To justify the usefulness of our criterion, we further verify it for a particular discrete-time Markov dynamical system. From the application point of view, the examined system provides a useful tool in analysing the stochastic dynamics of gene expression in prokaryotes.
This paper presents some new results on the conditional joint probability distributions of phase-type under the mixture of right-continuous Markov jump processes with absorption on the same finite state space $\mathbb{S}$ moving at different speeds, where the mixture occurs at a random time. Such mixture was first proposed by Frydman \cite{Frydman2005} and Frydman and Schuermann \cite{Frydman2008} as a generalization of the mover-stayer model of Blumen et at. \cite{Blumen}, and was recently extended by Surya \cite{Surya2018}. When conditioning on all previous and current information $\mathcal{…
This is an extended version of a series of lectures given in St Flour. It includes a discussion of relations between the occupation field of Markov loops with the corresponding free field.
New results on conditional joint probability distributions of first exit times are presented for a continuous-time stochastic process defined as the mixture of Markov jump processes moving at different speeds on the same finite state space, while the mixture occurs at a random time. Such mixture was first proposed by Frydman \cite{Frydman2005} and Frydman and Schuermann \cite{Frydman2008} as a generalization of the mover-stayer model of Blumen et at. \cite{Blumen}, and was recently extended by Surya \cite{Surya2018}, in which explicit distributional identities of the process are given, in part…
The present paper establishes a discrete version of the result obtained by P. Carr and S. Nadtochiy (2011) for 1-dimensional diffusion processes. Our result is for Markov chains on $\mathbf{Z}^d$.
In this note we show a simple formula for the joint density of local times, last exit tree and cycling numbers of continuous-time Markov Chains on finite graphs, which involves the modified Bessel function of the first type.
We prove existence of weak martingale solutions satisfying an almost sure version of the energy inequality and which constitute a (almost sure) Markov process.
In this paper we prove a version of the Berman-Konsowa principle for reversible Markov jump processes on Polish spaces. The Berman-Konsowa principle provides a variational formula for the capacity of a pair of disjoint measurable sets. There are two versions, one involving a class of probability measures for random finite paths from one set to the other, the other involving a class of finite unit flows from one set to the other. The Berman-Konsowa principle complements the Dirichlet principle and the Thomson principle, and turns out to be especially useful for obtaining sharp estimates on cros…
We prove a version of the multidimensional Fourth Moment Theorem for chaotic random vectors, in the general context of diffusion Markov generators. In addition to the usual componentwise convergence and unlike the infinite-dimensional Ornstein-Uhlenbeck generator case, another moment-type condition is required to imply joint convergence of of a given sequence of vectors.
This paper proposes additional material to the main statements of [HL13] which are are recalled in Section~2. In particular an application of [Theorem~2.2,HL13] to Renewal Markov Processes is provided in Section~4 and a detailed checking of the assumptions of [Theorem~2.2,HL13] for the joint distribution of local times of a finite jump process is reported in Section~5. A uniform version of [Theorem~2.2,HL13] with respect to a compact set of transition matrices is given in Section~6 (see [Remark 2.4,HL13]). The basic material on the semigroup of Fourier matrices and the spectral approach used i…
We present a Markov chain example where non-reversibility and an added edge jointly improve mixing time: when a random edge is added to a cycle of $n$ vertices and a Markov chain with a drift is introduced, we get mixing time of $O(n^{3/2})$ with probability bounded away from 0. If only one of the two modifications were performed, the mixing time would stay $Ω(n^2)$.
In this paper we study the distribution of the level at time $θ$ of Markovian fluid queues and Markovian continuous time random walks, the maximum (and minimum) level over $[0,θ]$, and their joint distributions. We approximate $θ$ by a random variable $T$ with Erlang distribution and we use an alternative way, with respect to the usual Laplace transform approach, to compute the distributions. We present probabilistic interpretation of the equations and provide a numerical illustration.
We define joint exchangeability on arrays indexed by a vector of natural numbers with coordinates being the vertices of directed acyclic graphs (DAGs) using local isomorphisms. The notion provides a new version of exchangeability, which is a joint version of hierarchical exchangeability defined in Jung, L., Staton, Yang (2020). We also prove the existence of a generic representation by independent uniform random variables.
The density-dependent Markov chain (DDMC) introduced in \cite{Kurtz1978} is a continuous time Markov process applied in fields such as epidemics, chemical reactions and so on. In this paper, we give moderate deviation principles of paths of DDMC under some generally satisfied assumptions. The proofs for the lower and upper bounds of our main result utilize an exponential martingale and a generalized version of Girsanov's theorem. The exponential martingale is defined according to the generator of DDMC.
We consider a continuous time Markov chain on a countable state space and prove a joint large deviation principle for the empirical measure and the empirical flow, which accounts for the total number of jumps between pairs of states. We give a direct proof using tilting and an indirect one by contraction from the empirical process.
In the last years, many authors studied a class of continuous time semi-Markov processes obtained by time-changing Markov processes by hitting times of independent subordinators. Such processes are governed by integro-differential convolution equations of generalized fractional type. The aim of this paper is to develop the discrete-time version of such a theory. We show that a class of discrete-time semi-Markov chains can be seen as time-changed Markov chains and we obtain governing convolution type equations. Such processes converge weakly to those in continuous time under suitable scaling li…
We formulate a probabilistic Markov property in discrete time under a dynamic risk framework with minimal assumptions. This is useful for recursive solutions to risk-sensitive versions of dynamic optimisation problems such as optimal prediction, where at each stage the recursion depends on the whole future. The property holds for standard measures of risk used in practice, and is formulated in several equivalent versions including a representation via acceptance sets, a strong version, and a dual representation.
We investigate the local times of a continuous-time Markov chain on an arbitrary discrete state space. For fixed finite range of the Markov chain, we derive an explicit formula for the joint density of all local times on the range, at any fixed time. We use standard tools from the theory of stochastic processes and finite-dimensional complex calculus. We apply this formula in the following directions: (1) we derive large deviation upper estimates for the normalized local times beyond the exponential scale, (2) we derive the upper bound in Varadhan's lemma for any measurable functional of the l…
In the present paper the infinite-server MMAPkGk queueing model with random resource vector of customers, marked MAP arrival and semi-Markov (SM) arrival of catastrophes is considered. The joint generating functions (PGF) of transient and stationary distributions of number of busy servers and numbers of different types served customers, as well as Laplace transformations (LT) of joint distributions of total accumulated resources in the model at moment and total accumulated resources of served customers during time interval are found. The basic differential and renewal equations for transient a…
We consider an irreducible continuous time Markov chain on a finite state space and with time periodic jump rates and prove the joint large deviation principle for the empirical measure and flow and the joint large deviation principle for the empirical measure and current. By contraction we get the large deviation principle of three types of entropy production flow. We derive some Gallavotti-Cohen duality relations and discuss some applications.
We consider the connections among `clumped' residual allocation models (RAMs), a general class of stick-breaking processes including Dirichlet processes, and the occupation laws of certain discrete space time-inhomogeneous Markov chains related to simulated annealing and other applications. An intermediate structure is introduced in a given RAM, where proportions between successive indices in a list are added or clumped together to form another RAM. In particular, when the initial RAM is a Griffiths-Engen-McCloskey (GEM) sequence and the indices are given by the random times that an auxiliary …
We introduce a relative notion of the 'big images and preimages'-property for random topological Markov chains. This then implies that a relative version of the Ruelle-Perron-Frobenius theorem holds with respect to summable and locally Hoelder continuous potentials.
We prove that under certain mild assumptions, the entropy rate of a hidden Markov chain, observed when passing a finite-state stationary Markov chain through a discrete-time continuous-output channel, is jointly analytic as a function of the input Markov chain parameters and the channel parameters. In particular, as consequences of the main theorems, we obtain analyticity for the entropy rate associated with representative channels: Cauchy and Gaussian.
We recover the Donsker-Varadhan large deviations principle (LDP) for the empirical measure of a continuous time Markov chain on a countable (finite or infinite) state space from the joint LDP for the empirical measure and the empirical flow proved in [2].
We state and prove a quantitative version of the bounded difference inequality for geometrically ergodic Markov chains. Our proof uses the same martingale decomposition as \cite{MR3407208} but, compared to this paper, the exact coupling argument is modified to fill a gap between the strongly aperiodic case and the general aperiodic case.
We consider a continuous time Markov chain on a countable state space. We prove a joint large deviation principle (LDP) of the empirical measure and current in the limit of large time interval. The proof is based on results on the joint large deviations of the empirical measure and flow obtained in \cite{BFG}. By improving such results we also show, under additional assumptions, that the LDP holds with the strong L^1 topology on the space of currents. We deduce a general version of the Gallavotti-Cohen (GC) symmetry for the current field and show that it implies the so-called fluctuation theor…
We consider triangular arrays of Markov chains that converge weakly to a diffusion process. Second order Edgeworth type expansions for transition densities are proved. The paper differs from recent results in two respects. We allow nonhomogeneous diffusion limits and we treat transition densities with time lag converging to zero. Small time asymptotics are motivated by statistical applications and by resulting approximations for the joint density of diffusion values at an increasing grid of points.
We develop Markov categories as a framework for synthetic probability and statistics, following work of Golubtsov as well as Cho and Jacobs. This means that we treat the following concepts in purely abstract categorical terms: conditioning and disintegration; various versions of conditional independence and its standard properties; conditional products; almost surely; sufficient statistics; versions of theorems on sufficient statistics due to Fisher--Neyman, Basu, and Bahadur. Besides the conceptual clarity offered by our categorical setup, its main advantage is that it provides a uniform trea…
