Matryoshka dolls, the traditional Russian nesting figurines, are known world-wide for each doll's encapsulation of a sequence of smaller dolls. In this paper, we identify a large class of Markov process whose moments are easy to compute by exploiting the structure of a new sequence of nested matrices we call Matryoshkhan matrices. We characterize the salient properties of Matryoshkhan matrices that allow us to compute these moments in closed form at a specific time without computing the entire path of the process. This speeds up the computation of the Markov process moments significantly in comparison to traditional differential equation methods, which we demonstrate through numerical experiments. Through our method, we derive explicit expressions for both transient and steady-state moments of this class of Markov processes. We demonstrate the applicability of this method through explicit examples such as shot-noise processes, growth-collapse processes, linear birth-death-immigration processes, and affine stochastic differential equations from the finance literature. We also show that we can derive explicit expressions for the self-exciting Hawkes process, for which finding closed form moment expressions has been an open problem since its introduction in 1971. In general, our techniques can be used for any Markov process for which the infinitesimal generator of an arbitrary polynomial is itself a polynomial of equal or lower order
Markovian maximal couplings of Markov processes are characterized by an equality of total variation and a distance of Wasserstein type. If a Markovian maximal coupling is a Feller process, the generator can be calculated, e.g. for reflection coupled Brownian motion. Apart from processes with continuous paths also jump processes are treated for the first time. For subordinated Brownian motion a Markovian maximal coupling is constructed by subordinating reflection coupled Brownian motion. This coupling is the unique Markovian maximal coupling and its generator is determined by state-space depend…
We consider the Markov chain approximations for singular stable-like processes. First we obtain properties of some Markov chains. Then we construct the approximating Markov chains and give a necessary condition for weak convergence of these chains to singular stable-like processes.
We present a class of Gauss-Markov processes which can be represented as space-time scaled stationary Ornstein-Uhlenbeck processes defined on the real line. We give several explicit examples of the representation for certain Gauss bridge processes. As an application, we derive a formula for the density function of the supremum location of certain standardized Gauss-Markov processes on compact time intervals. We also present some sufficient conditions under which mean centered Gauss-Markov processes take zero at a fixed time with probability one.
A variational formula for the asymptotic variance of general Markov processes is obtained. As application, we get a upper bound of the mean exit time of reversible Markov processes, and some comparison theorems between the reversible and non-reversible diffusion processes.
In this paper, we investigate spectral properties of explosive symmetric Markov processes. Under a condition on its life time, we prove the $L^1$-semigroup of Markov processes become compact operators.
A simple model of the new notion of "Markov up" processes is proposed; its positive recurrence and ergodic properties are shown under the appropriate conditions.
In the last years, many authors studied a class of continuous time semi-Markov processes obtained by time-changing Markov processes by hitting times of independent subordinators. Such processes are governed by integro-differential convolution equations of generalized fractional type. The aim of this paper is to develop the discrete-time version of such a theory. We show that a class of discrete-time semi-Markov chains can be seen as time-changed Markov chains and we obtain governing convolution type equations. Such processes converge weakly to those in continuous time under suitable scaling li…
Extensions of Kemeny's constant, as derived for irreducible finite Markov chains in discrete time, to Markov renewal processes and Markov chains in continuous time are discussed. Three alternative Kemeny's functions and their variants are considered. Typically, they lead to a constant if and only if the mean holding times between the states in the Markov renewal process are constant. However one particular variant leads to a constant, analogous to the discrete time Markov chain result. Specifically, if the state space is finite, the weighted sum of the mean first passage times (omitting the me…
In this paper we study Green measures for certain classes of random time change Markov processes where the random time change are inverse subordinators. We show the existence of the Green measure for these processes under the condition of the existence of the Green measure of the original Markov processes and they coincide. Applications to fractional dynamics in given.
We consider the down/up crossing property of weighted Markov branching processes. The joint probability distribution of multi crossing numbers of such processes are obtained. In particular, for Markov branching processes, the probability distribution of death number is given and for bulk-arrival queueing model, the joint probability distributions of service number and arrival number is also given.
Reinforced processes are known to provide a stochastic representation for the quasi-stationary distribution of a given killed Markov process - describing the killed Markov process at fixed time instants. In this paper we shall adapt the construction to provide a pathwise description. We also obtain a stochastic approximation for the quasi-limiting distributions of reducible killed Markov processes as a corollary.
In this paper, a class of piecewise deterministic Markov processes with underlying fast dynamic is studied. Using a "penalty method" , an averaging result is obtained when the underlying dynamic is infinitely accelerated. The features of the averaged process, which is still a piecewise deterministic Markov process, are fully described.
In this paper the class of mixed renewal processes (MRPs for short) with mixing parameter a random vector from \cite{lm6z3} (enlarging Huang's \cite{hu} original class) is replaced by the strictly more comprising class of all extended MRPs by adding a second mixing parameter. We prove under a mild assumption, that within this larger class the basic problem, whether every Markov process is a mixed Poisson process with a random variable as mixing parameter has a solution to the positive. This implies the equivalence of Markov processes, mixed Poisson processes, and processes with the multinomial…
Semi-Markov processes are a generalization of Markov processes since the exponential distribution of time intervals is replaced with an arbitrary distribution. This paper provides an integro-differential form of the Kolmogorov's backward equations for a large class of homogeneous semi-Markov processes, having the form of an abstract Volterra integro-differential equation. An equivalent evolutionary (differential) form of the equations is also provided. Fractional equations in the time variable are a particular case of our analysis. Weak limits of semi-Markov processes are also considered and t…
We study a class of Markov processes with finite state space and continuous time that have product form stationary distributions. We obtain a number of examples that can generate conjectures for diffusions with inert drift.
Necessary and sufficient conditions for convergence in distribution of first-rare-event times and convergence in Skorokhod J-topology of first-rare-event-time processes for perturbed semi-Markov processes with finite phase space are obtained.
We prove that the class of discrete time stationary max-stable process satisfying the Markov property is equal, up to time reversal, to the class of stationary max-autoregressive processes of order $1$. A similar statement is also proved for continuous time processes.
Measure-free discrete time stochastic processes in Riesz spaces were formulated and studied by Kuo, Labuschagne and Watson. Aspects relating martingales, stopping times, convergence of these processes as well as various decomposition were considered. Here we formulate and study Markov processes in a measure-free Riesz space setting.
In the development of stochastic integration and the theory of semimartingales, Markov processes have been a constant source of inspiration. Despite this historical interweaving, it turned out that semimartingales should be considered the `natural' class of processes for many concepts first developed in the Markovian framework. As an example, stochastic differential equations have been invented as a tool to study Markov processes but nowadays are treated separately in the literature. Moreover, the killing of processes has been known for decades before it made its way to the theory of semimarti…
Monotone Lévy processes with additive increments are defined and studied. It is shown that these processes have a natural Markov structure and their Markov transition semigroups are characterized using the monotone Lévy-Khintchine formula. Monotone Lévy processes turn out to be related to classical Lévy processes via Attal's ``remarkable transformation.'' A monotone analogue of the family of exponential martingales associated to a classical Lévy process is also defined.
In this paper we discuss weak convergence of continuous-time Markov chains to a non-symmetric pure jump process. We approach this problem using Dirichlet forms as well as semimartingales. As an application, we discuss how to approximate a given Markov process by Markov chains.
We present Lyapunov-type conditions for non-strong ergodicity of Markov processes. Some concrete models are discussed including diffusion processes on Riemannian manifolds and Ornstein-Uhlenbeck processes driven by symmetric $α$-stable processes. For SDE driven by $α$-stable process ($α\in (0,2]$) with polynomial drift, the strong ergodicity or not is independent on $α$.
New algorithms for computing of asymptotic expansions for stationary distributions of nonlinearly perturbed semi-Markov processes are presented. The algorithms are based on special techniques of sequential phase space reduction, which can be applied to processes with asymptotically coupled and uncoupled finite phase spaces.
We compute the Hamiltonian and Lagrangian associated to the large deviations of the trajectory of the empirical distribution for independent Markov processes, and of the empirical measure for translation invariant interacting Markov processes. We treat both the case of jump processes (continuous-time Markov chains and interacting particle systems) as well as diffusion processes. For diffusion processes, the Lagrangian is a quadratic form of the deviation of the trajectory from the Kolmogorov forward equation. In all cases, the Lagrangian can be interpreted as a relative entropy or relative ent…
We provide necessary and sufficient conditions for convergence of exponential integrals of Markov additive processes. Other than in the classical Lévy case studied by Erickson and Maller we have to distinguish between almost sure convergence and convergence in probability. Our proofs rely on recent results on perpetuities in a Markovian environment by Alsmeyer and Buckmann.
This paper aims to establish a central limit theorem for Markov processes conditioned not to be absorbed under a very general assumption on quasi-stationarity for the underlying process. To do so, a central limit theorem has been established for ergodic Markov processes. The conditional central limit theorem is then obtained by applying the central limit theorem to the $Q$-process.
We define analogues of Brownian motion on the triadic Cantor set by introducing a few natural requirements on the Markov semigroup. We give a detailed description of these symmetric self-similar processes and study their properties such as mixing and moment asymptotics.
We establish integral tests and laws of the iterated logarithm for the upper envelope of the future infimum of positive self-similar Markov processes and for increasing self-similar Markov processes at 0 and infinity. Our proofs are based on the Lamperti representation and time reversal arguments due to Chaumont and Pardo [9]. These results extend laws of the iterated logarithm for the future infimum of Bessel processes due to Khoshnevisan et al. [11].
Comparison results for Markov processes w.r.t. function class induced (integral) stochastic orders have a long history. The most general results so far for this problem have been obtained based on the theory of evolution systems on Banach spaces. In this paper we transfer the martingale comparison method, known for the comparison of semimartingales to Markovian semimartingales, to general Markov processes. The basic step of this martingale approach is the derivation of the supermartingale property of the linking process, giving a link between the processes to be compared. In this paper this pr…
We prove sharp estimates on heat kernels and Green functions for subordinate Markov processes with both discrete an continuous time, under relatively weak assumptions about original processes as well as Laplace exponents of subordinators. We also show stability of heat kernel estimates in the discrete settings.
We prove the Wiener-Hopf factorization for Markov Additive processes. We derive also Spitzer-Rogozin theorem for this class of processes which serves for obtaining Kendall's formula and Fristedt representation of the cumulant matrix of the ladder epoch process. Finally, we also obtain the so-called ballot theorem.
The goal of this paper is threefold. First, we survey the existing results on Hunt's hypothesis (H) for Markov processes and Getoor's conjecture for Lévy processes. Second, we investigate (H) for multidimensional Lévy processes from the viewpoints of projections and energy, respectively. Third, we present a few open questions for further study.
We give some relationships between the first Dirichlet eigenvalues and the exit time moments for the general symmetric Markov processes. As applications, we present some examples, including symmetric diffusions and $α$-stable processes, and provide the estimates of their first Dirichlet eigenvalues and the exit time moments.
We consider almost upper semi-continuous processes defined on a finite Markov chain. The distributions of the functionals associated with the exit from a finite interval are studied. We also consider some modification of these processes.
Processes having the same bridges are said to belong to the same reciprocal class. In this article we analyze reciprocal classes of Markov counting processes by identifying their reciprocal invariants and we characterize them as the set of counting processes satisfying some duality formula.
The coarse Ricci curvature for Markov chains is generalized for continuous time. We show that a positive coarse Ricci curvature implies a contraction of the Markov process for the Wasserstein distance between probability measures. This leads to a spectral gap estimate in the reversible case.
In this article, we define the new concept of local coupling property for Markov processes and study its relationship with distributional properties of the transition probability. In the special case of Lévy processes we show that this property is equivalent to the absolute continuity of the transition probability and also provide a sufficient condition for it in terms of the Lévy measure. Our result is stronger than existing results for absolute continuity of Lévy distributions.
We study the quasi-stationary behavior of multidimensional processes absorbed when one of the coordinates vanishes. Our results cover competitive or weakly cooperative Lotka-Volterra birth and death processes and Feller diffusions with competitive Lotka-Volterra interaction. To this aim, we develop original non-linear Lyapunov criteria involving two Lyapunov functions, which apply to general Markov processes.
Markov processes are popular mathematical models, studied by theoreticians for their intriguing properties, and applied by practitioners for their flexible structure. With this book we teach how to model and analyze Markov processes. We classify Markov processes based on their structural properties, which in turn determine which analytic methods are required for solving them. In doing so, we start in each chapter with specific examples that naturally lead up to general theory and general methods. In this way the reader learns about Markov processes on the job. By studying this book, the reader…
This paper provides a general and abstract approach to approximate ergodic regimes of Markov and Feller processes. More precisely, we show that the recursive algorithm presented in Lamberton & Pages (2002) and based on simulation algorithms of stochastic schemes with decreasing step can be used to build invariant measures for general Markov and Feller processes. We also propose applications in three different configurations: Approximation of Markov switching Brownian diffusion ergodic regimes using Euler scheme, approximation of Markov Brownian diffusion ergodic regimes with Milstein scheme an…
We establish integral tests and laws of the iterated logarithm for the lower envelope of positive self-similar Markov processes at 0 and $+\infty$. Our proofs are based on the Lamperti representation and time reversal arguments. These results extend laws of the iterated logarithm for Bessel processes due to Dvoretsky and Erdös, Motoo and Rivero.
Any exchangeable Markov processes on $[k]^{\mathbb{N}}$ with cadlag sample paths projects to a Markov process on the simplex whose sample paths are cadlag and of locally bounded variation. Furthermore, any such process has a de Finetti-type description as a mixture of i.i.d. copies of time-inhomogeneous Markov processes on $[k]$. In the Feller case, these time-inhomogeneous Markov processes have a relatively simple structure; however, in the non-Feller case a greater variety of behaviors is possible since the transition law of the underlying Markov process on $[k]^{\zz{N}}$ can depend in a non…
Since the seminal work of Lamperti there is a lot of interest in the understanding of the general structure of self-similar Markov processes. Lamperti gave a representation of positive self-similar Markov processes with initial condition strictly larger than 0 which subsequently was extended to zero initial condition. For real self-similar Markov processes (rssMps) there is a generalization of Lamperti's representation giving a one-to-one correspondence between Markov additive processes and rssMps with initial condition different from the origin. We develop fluctuation theory for Markov additi…
It was recently proven that the correlation function of the stationary version of a reflected Lévy process is nonnegative, nonincreasing and convex. In another branch of the literature it was established that the mean value of the reflected process starting from zero is nonnegative, nondecreasing and concave. In the present paper it is shown, by putting them in a common framework, that these results extend to substantially more general settings. Indeed, instead of reflected Lévy processes, we consider a class of more general stochastically monotone Markov processes. In this setup we show monot…
We provide an original and general sufficient criterion ensuring the exponential contraction of Feynman-Kac semi-groups of penalized processes. This criterion is applied to time-inhomogeneous one-dimensional diffusion processes conditioned not to hit 0 and to penalized birth and death processes evolving in a quenched random environment.
We present a tail inequality for suprema of empirical processes generated by variables with finite $ψ_α$ norms and apply it to some geometrically ergodic Markov chains to derive similar estimates for empirical processes of such chains, generated by bounded functions. We also obtain a bounded difference inequality for symmetric statistics of such Markov chains.
We develop a new methodology for the fluctuation theory of continuous-time skip-free Markov chains, extending the recent work of Choi and Patie [5] for discrete-time skip-free Markov chains. As the main application we use it to derive a full set of fluctuation identities regarding exiting a finite or infinite interval for Markov branching processes with immigration, thereby uncovering many new results for this classical family of continuous-time Markov chains. The theory also allows us to recover in a simple manner fluctuation identities for skip-free downward compound Poisson processes.
In his 1972 paper, John Lamperti characterized all positive self-similar Markov processes as time-changes of exponentials of Levy processes. In the past decade the problem of classifying all non-negative self-similar Markov processes that do not necessarily have zero as a trap has been solved gradually via connections to ladder height processes and excursion theory. Motivated by a recent article of Chaumont, Rivero, Panti we classify via jump-type SDEs the symmetric real-valued self-similar Markov processes that only decrease the absolute value by jumps and leave zero continuously. Our constru…
We show that an isotropic self-similar Markov process in ${\Bbb R}^d$ has a skew product structure if and only if its radial and angular parts do not jump at the same time.
An analogue of the classical Mecke formula for Poisson point processes is proved for the class of space-time STIT tessellation processes. From this key identity the Markov property of a class of associated random processes is derived. This in turn is used to determine the distribution of the number of internal vertices of the typical maximal tessellation segment.
We consider a type of Markov property for set-indexed processes which is satisfied by all processes with independent increments and which allows us to introduce a transition system theory leading to the construction of the process. A set-indexed generator is defined such that it completely characterizes the distribution of the process.
New algorithms for computing power moments of hitting times and accumulated rewards of hitting type for semi-Markov processes. The algorithms are based on special techniques of sequential phase space reduction and recurrence relations connecting moments of rewards. Applications are discussed as well as possible generalisations of presented results and results of experimental studies.
The aim of this paper is to prove stability of traveling waves for integro-differential equations connected with branching Markov processes. In other words, the limiting law of the left-most particle of a (time-continuous) branching Markov process with a Lévy non-branching part is demonstrated. The key idea is to approximate the branching Markov process by a branching random walk and apply the result of Aïdékon [1] on the limiting law of the latter one.
We present sufficient conditions, in terms of the jumping kernels, for two large classes of conservative Markov processes of pure-jump type to be purely discontinuous martingales with finite second moment. As an application, we establish the law of the iterated logarithm for sample paths of the associated processes.
