This paper describes the structure of solutions to Kolmogorov's equations for nonhomogeneous jump Markov processes and applications of these results to control of jump stochastic systems. These equations were studied by Feller (1940), who clarified in 1945 in the errata to that paper that some of its results covered only nonexplosive Markov processes. In this work, which is largely of a survey nature, the case of explosive processes is also considered. This paper is based on the invited talk presented by the authors at the conference "Chebyshev-200", and it describes the results of their joined studies with Manasa Mandava (1984-2019).
As well-known, transition probabilities of jump Markov processes satisfy Kolmogorov's backward and forward equations. In the seminal 1940 paper, William Feller investigated solutions of Kolmogorov's equations for jump Markov processes. Recently the authors solved the problem studied by Feller and showed that the minimal solution of Kolmogorov's backward and forward equations is the transition probability of the corresponding jump Markov process if the transition rate at each state is bounded. This paper presents more general results. For Kolmogorov's backward equation, the sufficient condition…
This paper studies three ways to construct a nonhomogeneous jump Markov process: (i) via a compensator of the random measure of a multivariate point process, (ii) as a minimal solution of the backward Kolmogorov equation, and (iii) as a minimal solution of the forward Kolmogorov equation. The main conclusion of this paper is that, for a given measurable transition intensity, commonly called a Q-function, all these constructions define the same transition function. If this transition function is regular, that is, the probability of accumulation of jumps is zero, then this transition function is…
We consider a stochastic functional delay differential equation, namely an equation whose evolution depends on its past history as well as on its present state, driven by a pure diffusive component plus a pure jump Poisson compensated measure. We lift the problem in the infinite dimensional space of square integrable Lebesgue functions in order to show that its solution is an $L^2-$valued Markov process whose uniqueness can be shown under standard assumptions of locally Lipschitzianity and linear growth for the coefficients. Coupling the aforementioned equation with a standard backward differe…
Semi-Markov processes are a generalization of Markov processes since the exponential distribution of time intervals is replaced with an arbitrary distribution. This paper provides an integro-differential form of the Kolmogorov's backward equations for a large class of homogeneous semi-Markov processes, having the form of an abstract Volterra integro-differential equation. An equivalent evolutionary (differential) form of the equations is also provided. Fractional equations in the time variable are a particular case of our analysis. Weak limits of semi-Markov processes are also considered and t…
Permanental sequences with non-symmetric kernels that are generalization of the potentials of a Markov chain with state space $\{0,1/2, \ldots, 1/n,\ldots\}$ that was introduced by Kolmogorov, are studied. Depending on a parameter in the kernels we obtain an exact rate of divergence of the sequence at $0$, an exact local modulus of continuity of the sequence at $0$, or a precise bounded discontinuity for the sequence at $0$.
We consider a Markov process on a Riemannian manifold, which solves a stochastic differential equation in the interior of the manifold and jumps according to a deterministic reset map when it reaches the boundary. We derive a partial differential equation for the probability density function, involving a non-local boundary condition which accounts for the jumping behaviour of the process. This is a generalisation of the usual Fokker-Planck-Kolmogorov equation for diffusion processes. The result is illustrated with an example in the field of stochastic hybrid systems.
In this paper we discuss weak convergence of continuous-time Markov chains to a non-symmetric pure jump process. We approach this problem using Dirichlet forms as well as semimartingales. As an application, we discuss how to approximate a given Markov process by Markov chains.
We prove that if we are given a generator of a cadlag Markov process and an open domain $G$ in the state space, on which the generator has the local property expressed in a suitable way on a class $\mathcal{C}$ of test functions that is sufficiently rich, then the Markov process has continuous paths when it passes through $G$. The result holds for any Markov process which is associated with the generator merely on $\mathcal{C}$. This points out that the path continuity of the process is an a priori property encrypted by the generator acting on enough test functions, and this property can be ea…
For a continuous-time Markov process, we characterize the law of the first jump location when started from an arbitrary initial distribution, in terms of the invariant distribution of an auxiliary Markov process. This could be of interest in the burgeoning fields of piecewise-deterministic Markov chain Monte Carlo methods and quasi-stationary Monte Carlo methods.
We present sufficient conditions, in terms of the jumping kernels, for two large classes of conservative Markov processes of pure-jump type to be purely discontinuous martingales with finite second moment. As an application, we establish the law of the iterated logarithm for sample paths of the associated processes.
In this paper, we are interested in the exact simulation of a class of Piecewise Deterministic Markov Processes (PDMP). We show how to perform efficient thinning algorithms depending on the jump rate bound. For different types of jump rate bounds, we compare theoretically the mean number of generated (total) jump times and we compare numerically the simulation times. We use thinning algorithms on Hodgkin-Huxley models with Markovian ion channels dynamic to illustrate the results.
Motivated by entropic optimal transport, time reversal of Markov jump processes in $\mathbb{R}^n$ is investigated. Relying on an abstract integration by parts formula for the carré du champ of a Markov process recently obtained by Cattiaux, Gentil and the authors, and using an entropic improvement strategy discovered by Föllmer in the eighties, we compute the semimartingale characteristics of the time reversed process for a wide class of jump processes with possibly unbounded variation sample paths and singular intensities of jump.
We consider piecewise deterministic Markov processes with degenerate transition kernels of the "house-of-cards"-type. We use a splitting scheme based on jump times to prove the absolute continuity, as well as some regularity, of the invariant measure of the process. Finally, we obtain finer results on the regularity of the one-dimensional marginals of the invariant measure, using integration by parts with respect to the jump times.
In this paper we calculate Kolmogorov-Sinai entropy $h_M(S)$ of the generalized Markov shift associated with a contractive Markov system (CMS) \cite{Wer1} using the coding map constructed in \cite{Wer3}. We show that \[h_M(S)=-\sum\limits_{e\in E}\int\limits_{K_{i(e)}} p_e\log p_edμ\] where $μ$ is a unique invariant Borel probability measure of the CMS. I. Werner, Contractive Markov systems, J. London Math. Soc. (2005) 236-258. I. Werner, Coding map for a contractive Markov system, Math. Proc. Camb. Phil. Soc. to appear 140 (2), March 2006.
We aim in proving Poincaré inequalities for a class of pure jump Markov processes inspired by the model introduced in \cite{G-L} by Galves and Löcherbach to describe the behaviour of interacting brain neurons. In particular, we consider neurons with degenerate jumps, i.e. that lose their memory when they spike, while the probability of a spike depends on the actual position and thus the past of the whole neural system.
In this note we show a simple formula for the joint density of local times, last exit tree and cycling numbers of continuous-time Markov Chains on finite graphs, which involves the modified Bessel function of the first type.
We study Talagrand concentration and Poincaré type inequalities for unbounded pure jump Markov processes. In particular we focus on processes with degenerate jumps that depend on the past of the whole system, based on the model introduced by Galves and Löcherbach in \cite{G-L}, in order to describe the activity of a biological neural network. As a result we obtain exponential rates of convergence to equilibrium.
In this paper we study backward stochastic differential equations (BSDEs) driven by the compensated random measure associated to a given pure jump Markov process X on a general state space K. We apply these results to prove well-posedness of a class of nonlinear parabolic differential equations on K, that generalize the Kolmogorov equation of X. Finally we formulate and solve optimal control problems for Markov jump processes, relating the value function and the optimal control law to an appropriate BSDE that also allows to construct probabilistically the unique solution to the Hamilton-Jacobi…
We study the rate of weak convergence of Markov chains to diffusion processes under suitable but quite general assumptions. We give an example in the financial framework, applying the convergence analysis to a multiple jumps tree approximation of the CIR process. Then, we combine the Markov chain approach with other numerical techniques in order to handle the different components in jump-diffusion coupled models. We study the speed of convergence of this hybrid approach and we provide an example in finance, applying our results to a tree-finite difference approximation in the Heston or Bates m…
We derive concentration inequalities for empirical means $\frac{1}{t} \int_0^t f(X_s) ds$ where $X_s$ is an irreducible Markov jump process on a finite state space and $f$ some observable. Using a Feynman-Kac semigroup we first derive a general concentration inequality. Then, based on this inequality we derive further concentration inequalities. Hereby we use three different methods; perturbation theory, Poincaré inequalities and information inequalities. We also obtain a Bernstein type concentration inequality.
In this paper, we obtain new drift-type conditions for nonexplosion and explosion for nonhomogeneous Markov pure jump processes in Borel state spaces. The conditions are sharp; e.g., the one for nonexplosion is necessary if the state space is in addition locally compact and the $Q$-function satisfies weak Feller-type and local boundedness conditions. We comment on the relations of our conditions with the existing ones in the literature, and demonstrate some possible applications.
We present a new approach to positive self-similar Markov processes (pssMps) by reformulating Lamperti's transformation via jump type SDEs. As applications, we give direct constructions of pssMps (re)started continuously at zero if the Lamperti transformed Levy process is spectrally negative. Our paper can be seen as a continuation of similar studies for continuous state branching processes but the approach seems to be more fruitful in the context of pssMps.
Consider jump-type stochastic differential equations with the drift, diffusion and jump terms. Logarithmic derivatives of densities for the solution process are studied, and the Bismut-Elworthy-Li type formulae can be obtained under the uniformly elliptic condition on the coefficients of the diffusion and jump terms. Our approach is based upon the Kolmogorov backward equation by making full use of the Markovian property of the process.
We start by providing an explicit characterization and analytical properties, including the persistence phenomena, of the distribution of the extinction time $\mathbb{T}$ of a class of non-Markovian self-similar stochastic processes with two-sided jumps that we introduce as a stochastic time-change of Markovian self-similar processes. For a suitably chosen time-changed, we observe, for classes with two-sided jumps, the following surprising facts. On the one hand, all the $\mathbb{T}$'s within a class have the same law which we identify in a simple form for all classes and reduces, in the spect…
In his 1972 paper, John Lamperti characterized all positive self-similar Markov processes as time-changes of exponentials of Levy processes. In the past decade the problem of classifying all non-negative self-similar Markov processes that do not necessarily have zero as a trap has been solved gradually via connections to ladder height processes and excursion theory. Motivated by a recent article of Chaumont, Rivero, Panti we classify via jump-type SDEs the symmetric real-valued self-similar Markov processes that only decrease the absolute value by jumps and leave zero continuously. Our constru…
A systematic exposition of scale functions is given for positive self-similar Markov processes (pssMp) with one-sided jumps. The scale functions express as convolution series of the usual scale functions associated with spectrally one-sided Lévy processes that underly the pssMp through the Lamperti transform. This theory is then brought to bear on solving the spatio-temporal: (i) two-sided exit problem; (ii) joint first passage problem upwards for the the pssMp and its multiplicative drawdown (resp. drawup) in the spectrally negative (resp. positive) case.
We investigate a piecewise-deterministic Markov process, evolving on a Polish metric space, whose deterministic behaviour between random jumps is governed by some semi-flow, and any state right after the jump is attained by a randomly selected continuous transformation. It is assumed that the jumps appear at random moments, which coincide with the jump times of a Poisson process with intensity $λ$. The model of this type, although in a more general version, was examined in our previous papers, where we have shown, among others, that the Markov process under consideration possesses a unique inv…
A general formulation of the Fokker-Planck-Kolmogorov (FPK) equation for stochastic hybrid systems is presented, within the framework of Generalized Stochastic Hybrid Systems (GSHS). The FPK equation describes the time evolution of the probability law of the hybrid state. Our derivation is based on the concept of mean jump intensity, which is related to both the usual stochastic intensity (in the case of spontaneous jumps) and the notion of probability current (in the case of forced jumps). This work unifies all previously known instances of the FPK equation for stochastic hybrid systems, and …
The focus of this work is on local stability of a class of nonlinear ordinary differential equations (ODE) that describe limits of empirical measures associated with finite-state weakly interacting N-particle systems. Local Lyapunov functions are identified for several classes of such ODE, including those associated with systems with slow adaptation and Gibbs systems. Using results from [5] and large deviations heuristics, a partial differential equation (PDE) associated with the nonlinear ODE is introduced and it is shown that positive definite subsolutions of this PDE serve as local Lyapunov…
Let $f$ be the density function associated to a matrix-exponential distribution of parameters $(α, T,s)$. By exponentially tilting $f$, we find a probabilistic interpretation which generalises the one associated to phase-type distributions. More specifically, we show that for any sufficiently large $λ\ge 0$, the function $x\mapsto \left(\int_0^\infty e^{-λr}f(r) dr\right)^{-1}e^{-λx}f(x)$ can be described in terms of a Markov jump process whose generator is tied to $T$. Finally, we show how to revert the exponential tilting in order to assign a probabilistic interpretation to $f$ itself.
The law of the iterated logarithm for a piecewise deterministic Markov process assured by the properties of the Markov chain given by its post-jump locations
We show that the convergence of finite state space Markov chains to stationarity can often be considerably speeded up by alternating every step of the chain with a deterministic move. Under fairly general conditions, we show that not only do such schemes exist, they are numerous.
The purpose of this paper is to consider the exit-time problem for a finite-range Markov jump process, i.e, the distance the particle can jump is bounded independent of its location. Such jump diffusions are expedient models for anomalous transport exhibiting super-diffusion or nonstandard normal diffusion. We refer to the associated deterministic equation as a volume-constrained nonlocal diffusion equation. The volume constraint is the nonlocal analogue of a boundary condition necessary to demonstrate that the nonlocal diffusion equation is well-posed and is consistent with the jump process. …
"Quantum trajectories" are solutions of stochastic differential equations also called Belavkin or Stochastic Schrödinger Equations. They describe random phenomena in quantum measurement theory. Two types of such equations are usually considered, one is driven by a one-dimensional Brownian motion and the other is driven by a counting process. In this article, we present a way to obtain more advanced models which use jump-diffusion stochastic differential equations. Such models come from solutions of martingale problems for infinitesimal generators. These generators are obtained from the limit o…
This paper concerns the Vertex Reinforced Jump Process (VRJP) and its representations as a Markov process in random environment. We show that all possible representations of the VRJP as a mixture of Markov processes can be expressed in a similar form, using a random potential and harmonic functions for an associated operator. This allows to show that the VRJP on $\mathbb{Z}^d$ (with certain initial conditions) has a unique representation, by proving that an associated Martin boundary is trivial. Moreover, on infinite trees, we construct a family of representations, that are all different when …
We prove existence of boundary limits of ratios of positive harmonic functions for a wide class of Markov processes with jumps and irregular domains, in the context of general metric measure spaces. As a corollary, we prove uniqueness of the Martin kernel at each boundary point, that is, we identify the Martin boundary with the topological boundary. We also prove a Martin representation theorem for harmonic functions. Examples covered by our results include: strictly stable Lévy processes in R^d with positive continuous density of the Lévy measure; stable-like processes in R^d and in domains; …
Mean-field integro-differential equations are studied in an abstract framework, through couplings of the corresponding stochastic processes. In the perturbative regime, the equation is proven to admit a unique equilibrium, toward which the process converges exponentially fast. Similarly, in this case, the associated particle system is proven to converge toward its equilibrium at a rate independent from the number of particles.
Consider a system of interacting particles indexed by the nodes of a graph whose vertices are equipped with marks representing parameters of the model such as the environment or initial data. Each particle takes values in a countable state space and evolves according to a (possibly non-Markovian) continuous-time pure jump process whose jump intensities depend only on its own state (or history) and marks as well as the states (or histories) and marks of particles and edges in its neighborhood in the graph. Under mild conditions on the jump intensities, it is shown that the trajectories of the i…
Consider on a manifold the solution $X$ of a stochastic differential equation driven by a Lévy process without Brownian part. Sufficient conditions for the smoothness of the law of $X_t$ are given, with particular emphasis on noncompact manifolds. The result is deduced from the case of affine spaces by means of a localisation technique. The particular cases of Lie groups and homogeneous spaces are discussed.
We consider a class of continuous time Markov chains on a compact metric space that admit an invariant measure strictly positive on open sets together with absorbing states. We prove the joint large deviation principle for the empirical measure and flow. Due to the lack of uniform ergodicity, the zero level set of the rate function is not a singleton. As corollaries, we obtain the Donsker-Varadhan rate function for the empirical measure and a variational expression of the rate function for the empirical flow.
In this study, we consider a class of backward SDE driven by jump Markov process. An existence and uniqueness result to this kind of equations is obtained in a locally Lipschitz case. We essentially approximate the initial problem by constructing a convenient sequence of globally Lipschitz BSDEs having the existence and the uniqueness propriety. Then, we show, by passing to the limits, the existence, and uniqueness of a solution to the initial problem. After that, a stability theorem is also proved in the local Lipschitz setting. Applying the aforementioned result, we give an application to Eu…
The large deviations principle for the empirical measure for both continuous and discrete time Markov processes is well known. Various expressions are available for the rate function, but these expressions are usually as the solution to a variational problem, and in this sense not explicit. An interesting class of continuous time, reversible processes was identified in the original work of Donsker and Varadhan for which an explicit expression is possible. While this class includes many (reversible) processes of interest, it excludes the case of continuous time pure jump processes, such as a re…
In this paper we prove a version of the Berman-Konsowa principle for reversible Markov jump processes on Polish spaces. The Berman-Konsowa principle provides a variational formula for the capacity of a pair of disjoint measurable sets. There are two versions, one involving a class of probability measures for random finite paths from one set to the other, the other involving a class of finite unit flows from one set to the other. The Berman-Konsowa principle complements the Dirichlet principle and the Thomson principle, and turns out to be especially useful for obtaining sharp estimates on cros…
This paper considers adiabatic reduction in a model of stochastic gene expression with bursting transcription considered as a jump Markov process. In this model, the process of gene expression with auto-regulation is described by fast/slow dynamics. The production of mRNA is assumed to follow a compound Poisson process occurring at a rate depending on protein levels (the phenomena called bursting in molecular biology) and the production of protein is a linear function of mRNA numbers. When the dynamics of mRNA is assumed to be a fast process (due to faster mRNA degradation than that of protein…
We study a one-dimensional Markov modulated random walk with jumps. It is assumed that amplitudes of jumps as well as a chosen velocity regime are random and depend on a time spent by the process at a previous state of the underlying Markov process. Equations for the distribution and equations for its moments are derived. We characterise the martingale distributions in terms of observable proportions between jump and velocity regimes.
For integer valued random variables, the translated Poisson distributions form a flexible family for approximation in total variation, in much the same way that the normal family is used for approximation in Kolmogorov distance. Using the Stein--Chen method, approximation can often be achieved with error bounds of the same order as those for the CLT. In this paper, an analogous theory, again based on Stein's method, is developed in the multivariate context. The approximating family consists of the equilibrium distributions of a collection of Markov jump processes, whose analogues in one dimens…
We review some developments concerning Markov and Feller processes with jumps in geometric settings. These include stochastic differential equations in Markus canonical form, the Courrège theorem on Lie groups, and invariant Markov processes on manifolds under both transitive and more general Lie group actions.
We study optimal investment strategies that maximize expected utility from consumption and terminal wealth in a pure-jump asset price model with Markov-modulated (regime switching) jump-size distributions. We give sufficient conditions for existence of optimal policies and find closed-form expressions for the optimal value function for agents with logarithmic and fractional power (CRRA) utility in the case of two-state Markov chains. The main tools are convex duality techniques, stochastic calculus for pure-jump processes and explicit formulae for the moments of telegraph processes with Markov…
This paper has been withdrawn from the arXiv. It is now published by Elsevier in Nonlinear Analysis: Hybrid Systems, see http://dx.doi.org/10.1016/j.nahs.2009.07.008 . A general formulation of the Fokker-Planck-Kolmogorov (FPK) equation for stochastic hybrid systems is presented, within the framework of Generalized Stochastic Hybrid Systems (GSHS). The FPK equation describes the time evolution of the probability law of the hybrid state. Our derivation is based on the concept of mean jump intensity, which is related to both the usual stochastic intensity (in the case of spontaneous jumps) and t…
