We here study random evolutions on Banach spaces, driven by a class of semi-Markov processes. The expectation (in the sense of Bochner) of such evolutions is shown to solve some abstract Cauchy problems. Further, the abstract telegraph (damped wave) equation is generalized to the case of semi-Markov perturbations. A special attention is devoted to semi-Markov models of scattering transport processes which can be represented through these evolutions. In particular, we consider random flights with infinite mean flight times which turn out to be governed by a semi-Markov generalization of a linear Boltzmann equation; their scaling limit is proved to converge to superdiffusive transport processes.
In this short note we consider semi-Markov processes satisfying the condition of direction-time independence (Markov renewal processes). We derive large deviation principles and fluctuation theorems for the empirical current and the empirical currents along cycles. Our derivation is based on the joint LDP for the empirical measure and flow recently proved in \cite{MZ}.
We consider a finite state discrete time process X. Without loss of generality the finite state space can be identified with the set of unit vectors {e1, e2, . . . , eN} with ei = (0, . . . , 0, 1, 0, . . . , 0)0 2 RN. For a Markov chain the times the process stays in any state are geometrically distributed. This condition is relaxed for a semi-Markov chain. We first derive the semimartingale dynamics for a semi-Markov chain. We then consider the situation where the chain is observed in noise. We suggest how to estimate the occupation times in the states and derive filters and smoothers for qu…
In this paper, we investigate spectral properties of explosive symmetric Markov processes. Under a condition on its life time, we prove the $L^1$-semigroup of Markov processes become compact operators.
A quantum Markov semigroup can be represented via classical diffusion processes solving a stochastic Schrödinger equation. In this paper we first prove that a quantum Markov semigroup is irreducible if and only if classical diffusion processes are total in the Hilbert space of the system. Then we study the relationship between irreducibility of a quantum Markov semigroup and properties of these diffusions such as accessibility, the Lie algebra rank condition, and irreducibility. We prove that all these properties are, in general, weaker than irreducibility of the quantum Markov semigroup, neve…
Semi-Markov processes are a generalization of Markov processes since the exponential distribution of time intervals is replaced with an arbitrary distribution. This paper provides an integro-differential form of the Kolmogorov's backward equations for a large class of homogeneous semi-Markov processes, having the form of an abstract Volterra integro-differential equation. An equivalent evolutionary (differential) form of the equations is also provided. Fractional equations in the time variable are a particular case of our analysis. Weak limits of semi-Markov processes are also considered and t…
In the development of stochastic integration and the theory of semimartingales, Markov processes have been a constant source of inspiration. Despite this historical interweaving, it turned out that semimartingales should be considered the `natural' class of processes for many concepts first developed in the Markovian framework. As an example, stochastic differential equations have been invented as a tool to study Markov processes but nowadays are treated separately in the literature. Moreover, the killing of processes has been known for decades before it made its way to the theory of semimarti…
We consider a class of semi-Markov processes (SMP) such that the embedded discrete time Markov chain may be non-homogeneous. The corresponding augmented processes are represented as semi-martingales using stochastic integral equation involving a Poisson random measure. The existence and uniqueness of the equation are established. Subsequently, we show that the solution is indeed a SMP with desired transition rate. Finally, we derive the law of the bivariate process obtained from two solutions of the equation having two different initial conditions.
Necessary and sufficient conditions for convergence in distribution of first-rare-event times and convergence in Skorokhod J-topology of first-rare-event-time processes for perturbed semi-Markov processes with finite phase space are obtained.
In the last years, many authors studied a class of continuous time semi-Markov processes obtained by time-changing Markov processes by hitting times of independent subordinators. Such processes are governed by integro-differential convolution equations of generalized fractional type. The aim of this paper is to develop the discrete-time version of such a theory. We show that a class of discrete-time semi-Markov chains can be seen as time-changed Markov chains and we obtain governing convolution type equations. Such processes converge weakly to those in continuous time under suitable scaling li…
New algorithms for computing of asymptotic expansions for stationary distributions of nonlinearly perturbed semi-Markov processes are presented. The algorithms are based on special techniques of sequential phase space reduction, which can be applied to processes with asymptotically coupled and uncoupled finite phase spaces.
New algorithms for computing power moments of hitting times and accumulated rewards of hitting type for semi-Markov processes. The algorithms are based on special techniques of sequential phase space reduction and recurrence relations connecting moments of rewards. Applications are discussed as well as possible generalisations of presented results and results of experimental studies.
The main objective of this work is to present a process to compute the Markov renewal matrix for Markov renewal processes with countable infinite spaces, which semi-Markov matrixes are immigration and death type and assume a tridiagonal form. These processes occur often in practical applications. And the difficulty in obtaining friendly results for that matrix is a great obstacle to its application in practical cases modelling. It is considered the application to the MMinf queue particular case.
In this paper, we consider a class of inhomogeneous semi-Markov processes directly based on intensity processes for marked point processes. We show that this class satisfies the semi-Markov properties defined elsewhere in the literature. We use the marked point process setting to derive strong upper bounds on various probabilities for semi-Markov processes. Using these bounds, we rigorously prove for the case of countably infinite state space that the transition intensities are right-derivatives of the transition probabilities, and we prove for the case of finite state space that the transitio…
The aim of this note is to show, by providing an elementary way to construct counter-examples, that the strong Feller and the joint (space-time) continuity for a semigroup of Markov kernels on a Polish space are not enough to ensure the existence of an associated càdlàg Markov process on the same space. One such simple counter-example is the Brownian semigroup on $\mathbb{R}$ restricted to $\mathbb{R}\setminus \{0\}$, for which it is shown that there is no associated càdlàg Markov process. Using the same idea and results from potential theory we then prove that the analogous result with càdlàg…
The paper presents new asymptotic recurrent algorithms of phase space reduction for regularly and singularly perturbed semi-Markov processes. These algorithms give effective conditions of weak convergence for distributions and convergence of expectations for hitting times as well as recurrent formulas for computing the corresponding normalisation functions, Laplace transforms for limiting distributions and limits for expectations.
The mathematics of the finite single server queue with Poisson input and semi-Markov service times($M/SM/1/b$) is similar to that used for $BMAP/G/1/b$ systems. This observation results in new analytical formulas for a queue size in the $M/SM/1/b$ system. Both stationary and the transient solutions are considered
The goal of the paper is to describe the large time behaviour of a Markov process associated with a symmetric diffusion in a high-contrast random environment and to characterize the limit semigroup and the limit process under the diffusive scaling.
In this paper we characterize the distribution of the first exit time from an arbitrary open set for a class of semi-Markov processes obtained as time-changed Markov processes. We estimate the asymptotic behaviour of the survival function (for large $t$) and of the distribution function (for small $t$) and we provide some conditions for absolute continuity. We have been inspired by a problem of neurophyshiology and our results are particularly usefull in this field, precisely for the so-called Leacky Integrate-and-Fire (LIF) models: the use of semi-Markov processes in these models appear to be…
We consider a discrete time semi-Markov process where the characteristics defining the process depend on a small perturbation parameter. It is assumed that the state space consists of one finite communicating class of states and, in addition, one absorbing state. Our main object of interest is the asymptotic behaviour of the joint probabilities of the position of the semi-Markov process and the event of non-absorption as time tends to infinity and the perturbation parameter tends to zero. The main result gives exponential expansions of these probabilities together with an recursive algorithm f…
In this paper we construct a conservative Markov semi-group with generator $L=Δ+b\cdot\nabla$ on $\mathbb{R}^n$, where $b$ is a divergence-free vector field which belongs to $L^{2}\cap L^{p}$ with $\frac{n}{2}<p$. The research is motivated by the question of understanding the blow-up solutions of the fluid dynamic equations, which attracts a lot of attention in recent years.
Exponential relaxation to equilibrium is a typical property of physical systems, but inhomogeneities are known to distort the exponential relaxation curve, leading to a wide variety of relaxation patterns. Power law relaxation is related to fractional derivatives in the time variable. More general relaxation patterns are considered here, and the corresponding semi-Markov processes are studied. Our method, based on Bernstein functions, unifies three different approaches in the literature.
In this paper, we study quasi-stationary distributions of nonlinearly perturbed semi-Markov processes in discrete time. This type of distributions is of interest for the analysis of stochastic systems which have finite lifetimes, but are expected to persist for a long time. We obtain asymptotic power series expansions for quasi-stationary distributions and it is shown how the coefficients in these expansions can be computed from a recursive algorithm. As an illustration of this algorithm, we present a numerical example for a discrete time Markov chain.
In this paper, we study mixed power-exponential moment functionals of nonlinearly perturbed semi-Markov processes in discrete time. Conditions under which the moment functionals of interest can be expanded in asymptotic power series with respect to the perturbation parameter are given. We show how the coefficients in these expansions can be computed from explicit recursive formulas. In particular, the results of the present paper have applications for studies of quasi-stationary distributions.
Asymptotic expansions with explicit upper bounds for remainders are given for stationary distributions of nonlinearly perturbed semi-Markov processes with finite phase spaces. The corresponding algorithms are based on a special technique of sequential phase space reduction, which can be applied to processes with an arbitrary asymptotic communicative structure of phase spaces.
We provide a abstract framework to prove total variation convergence result with arbitrary rate for numerical scheme for SDE. In particular we show that under standard weak approximation properties of scheme such as Euler we can obtain total variation convergence with any desired rate by building a specific approximation for the SDE.
New algorithms for construction of asymptotic expansions for stationary distributions of nonlinearly perturbed semi-Markov processes with finite phase spaces are presented. These algorithms are based on a special technique of sequential phase space reduction, which can be applied to processes with an arbitrary asymptotic communicative structure of phase spaces. Asymptotic expansions are given in two forms, without and with explicit bounds for remainders.
The deterministic analog of the Markov property of a time-homogeneous Markov process is the semigroup property of solutions of an autonomous differential equation. The semigroup property arises naturally when the solutions of a differential equation are unique, and leads to a semiflow. We prove an abstract result on measurable selection of a semiflow for the situations without uniqueness. We outline applications to ODEs, PDEs, differential inclusions, etc. Our proof of the semiflow selection theorem is motivated by N. V. Krylov's Markov selection theorem. To accentuate this connection, we incl…
We extend the class of semimartingales in a natural way. This allows us to incorporate processes having paths that leave the state space R^d. In particular Markov processes related to sub-Markovian kernels, but also non-Markovian processes with path-dependent behavior. By carefully distinguishing between two killing states, we are able to introduce a fourth semimartingale characteristic which generalizes the fourth part of the Lévy quadruple. Using the probabilistic symbol, we analyze the close relationship between the generators of certain Markov processes with killing and their (now four) se…
In this paper we introduce a new approach to discrete-time semi-Markov decision processes based on the sojourn time process. Different characterizations of discrete-time semi-Markov processes are exploited and decision processes are constructed by their means. With this new approach, the agent is allowed to consider different actions depending also on the sojourn time of the process in the current state. A numerical method based on $Q$-learning algorithms for finite horizon reinforcement learning and stochastic recursive relations is investigated. Finally, we consider two toy examples: one in …
