In this paper, we outline a model of graph (or network) dynamics based on two ingredients. The first ingredient is a Markov chain on the space of possible graphs. The second ingredient is a semi-Markov counting process of renewal type. The model consists in subordinating the Markov chain to the semi-Markov counting process. In simple words, this means that the chain transitions occur at random time instants called epochs. The model is quite rich and its possible connections with algebraic geometry are briefly discussed. Moreover, for the sake of simplicity, we focus on the space of undirected graphs with a fixed number of nodes. However, in an example, we present an interbank market model where it is meaningful to use directed graphs or even weighted graphs.
Regular and singular parts of asymptotic expansions of semi-Markov random evolutions are given. Regularity of boundary conditions is shown. An algorithm for calculation of initial conditions is proposed.
We consider a finite state discrete time process X. Without loss of generality the finite state space can be identified with the set of unit vectors {e1, e2, . . . , eN} with ei = (0, . . . , 0, 1, 0, . . . , 0)0 2 RN. For a Markov chain the times the process stays in any state are geometrically distributed. This condition is relaxed for a semi-Markov chain. We first derive the semimartingale dynamics for a semi-Markov chain. We then consider the situation where the chain is observed in noise. We suggest how to estimate the occupation times in the states and derive filters and smoothers for qu…
In this paper, the weak convergence of impulsive recurrent process with semi-Markov switching in the scheme of Levy approximation is proved. Singular perturbation problem for the compensating operator of the extended Markov renewal process is used to prove the relative compactness.
Semi-Markov processes are a generalization of Markov processes since the exponential distribution of time intervals is replaced with an arbitrary distribution. This paper provides an integro-differential form of the Kolmogorov's backward equations for a large class of homogeneous semi-Markov processes, having the form of an abstract Volterra integro-differential equation. An equivalent evolutionary (differential) form of the equations is also provided. Fractional equations in the time variable are a particular case of our analysis. Weak limits of semi-Markov processes are also considered and t…
We consider a class of semi-Markov processes (SMP) such that the embedded discrete time Markov chain may be non-homogeneous. The corresponding augmented processes are represented as semi-martingales using stochastic integral equation involving a Poisson random measure. The existence and uniqueness of the equation are established. Subsequently, we show that the solution is indeed a SMP with desired transition rate. Finally, we derive the law of the bivariate process obtained from two solutions of the equation having two different initial conditions.
Necessary and sufficient conditions for convergence in distribution of first-rare-event times and convergence in Skorokhod J-topology of first-rare-event-time processes for perturbed semi-Markov processes with finite phase space are obtained.
In the last years, many authors studied a class of continuous time semi-Markov processes obtained by time-changing Markov processes by hitting times of independent subordinators. Such processes are governed by integro-differential convolution equations of generalized fractional type. The aim of this paper is to develop the discrete-time version of such a theory. We show that a class of discrete-time semi-Markov chains can be seen as time-changed Markov chains and we obtain governing convolution type equations. Such processes converge weakly to those in continuous time under suitable scaling li…
New algorithms for computing of asymptotic expansions for stationary distributions of nonlinearly perturbed semi-Markov processes are presented. The algorithms are based on special techniques of sequential phase space reduction, which can be applied to processes with asymptotically coupled and uncoupled finite phase spaces.
We consider almost upper semi-continuous processes defined on a finite Markov chain. The distributions of the functionals associated with the exit from a finite interval are studied. We also consider some modification of these processes.
New algorithms for computing power moments of hitting times and accumulated rewards of hitting type for semi-Markov processes. The algorithms are based on special techniques of sequential phase space reduction and recurrence relations connecting moments of rewards. Applications are discussed as well as possible generalisations of presented results and results of experimental studies.
The main objective of this work is to present a process to compute the Markov renewal matrix for Markov renewal processes with countable infinite spaces, which semi-Markov matrixes are immigration and death type and assume a tridiagonal form. These processes occur often in practical applications. And the difficulty in obtaining friendly results for that matrix is a great obstacle to its application in practical cases modelling. It is considered the application to the MMinf queue particular case.
In this paper, we consider a class of inhomogeneous semi-Markov processes directly based on intensity processes for marked point processes. We show that this class satisfies the semi-Markov properties defined elsewhere in the literature. We use the marked point process setting to derive strong upper bounds on various probabilities for semi-Markov processes. Using these bounds, we rigorously prove for the case of countably infinite state space that the transition intensities are right-derivatives of the transition probabilities, and we prove for the case of finite state space that the transitio…
The aim of this note is to show, by providing an elementary way to construct counter-examples, that the strong Feller and the joint (space-time) continuity for a semigroup of Markov kernels on a Polish space are not enough to ensure the existence of an associated càdlàg Markov process on the same space. One such simple counter-example is the Brownian semigroup on $\mathbb{R}$ restricted to $\mathbb{R}\setminus \{0\}$, for which it is shown that there is no associated càdlàg Markov process. Using the same idea and results from potential theory we then prove that the analogous result with càdlàg…
The paper presents new asymptotic recurrent algorithms of phase space reduction for regularly and singularly perturbed semi-Markov processes. These algorithms give effective conditions of weak convergence for distributions and convergence of expectations for hitting times as well as recurrent formulas for computing the corresponding normalisation functions, Laplace transforms for limiting distributions and limits for expectations.
The mathematics of the finite single server queue with Poisson input and semi-Markov service times($M/SM/1/b$) is similar to that used for $BMAP/G/1/b$ systems. This observation results in new analytical formulas for a queue size in the $M/SM/1/b$ system. Both stationary and the transient solutions are considered
We consider the almost semi-continuous processes defined on a finite Markov chain. The representation of the moment generating functions for the absolute maximum after achievement positive level and for the recovery time are obtained. Modified processes with two-step rate of negative jumps are investigated.
In this paper we characterize the distribution of the first exit time from an arbitrary open set for a class of semi-Markov processes obtained as time-changed Markov processes. We estimate the asymptotic behaviour of the survival function (for large $t$) and of the distribution function (for small $t$) and we provide some conditions for absolute continuity. We have been inspired by a problem of neurophyshiology and our results are particularly usefull in this field, precisely for the so-called Leacky Integrate-and-Fire (LIF) models: the use of semi-Markov processes in these models appear to be…
In this article almost semi-continuous processes with stationary independent increments on a finite irreducible Markov chain are considered. For these processes the components of matrix factorization identity are concretely defined. On the basis of this concrete definition the relations for the distributions of extrema and distributions of their complements for the almost upper semi-continuous processes are established.
We consider a discrete time semi-Markov process where the characteristics defining the process depend on a small perturbation parameter. It is assumed that the state space consists of one finite communicating class of states and, in addition, one absorbing state. Our main object of interest is the asymptotic behaviour of the joint probabilities of the position of the semi-Markov process and the event of non-absorption as time tends to infinity and the perturbation parameter tends to zero. The main result gives exponential expansions of these probabilities together with an recursive algorithm f…
In this paper we construct a conservative Markov semi-group with generator $L=Δ+b\cdot\nabla$ on $\mathbb{R}^n$, where $b$ is a divergence-free vector field which belongs to $L^{2}\cap L^{p}$ with $\frac{n}{2}<p$. The research is motivated by the question of understanding the blow-up solutions of the fluid dynamic equations, which attracts a lot of attention in recent years.
Exponential relaxation to equilibrium is a typical property of physical systems, but inhomogeneities are known to distort the exponential relaxation curve, leading to a wide variety of relaxation patterns. Power law relaxation is related to fractional derivatives in the time variable. More general relaxation patterns are considered here, and the corresponding semi-Markov processes are studied. Our method, based on Bernstein functions, unifies three different approaches in the literature.
In this paper, we study quasi-stationary distributions of nonlinearly perturbed semi-Markov processes in discrete time. This type of distributions is of interest for the analysis of stochastic systems which have finite lifetimes, but are expected to persist for a long time. We obtain asymptotic power series expansions for quasi-stationary distributions and it is shown how the coefficients in these expansions can be computed from a recursive algorithm. As an illustration of this algorithm, we present a numerical example for a discrete time Markov chain.
In this paper, we study mixed power-exponential moment functionals of nonlinearly perturbed semi-Markov processes in discrete time. Conditions under which the moment functionals of interest can be expanded in asymptotic power series with respect to the perturbation parameter are given. We show how the coefficients in these expansions can be computed from explicit recursive formulas. In particular, the results of the present paper have applications for studies of quasi-stationary distributions.
Asymptotic expansions with explicit upper bounds for remainders are given for stationary distributions of nonlinearly perturbed semi-Markov processes with finite phase spaces. The corresponding algorithms are based on a special technique of sequential phase space reduction, which can be applied to processes with an arbitrary asymptotic communicative structure of phase spaces.
Generators of Markov processes on a countable state space can be represented as finite or infinite matrices. One key property is that the off-diagonal entries corresponding to jump rates of the Markov process are non-negative. Here we present stochastic characterizations of the semigroup generated by a generator with possibly negative rates. This is done by considering a larger state space with one or more particles and antiparticles, with antiparticles being particles carrying a negative sign.
In this paper we propose a semi-Markov modulated model of interest rates. We assume that the switching process is a semi-Markov process with finite state space E and the modulated process is a diffusive process. We derive recursive equations for the higher order moments of the discount factor and we describe a Monte Carlo al- gorithm to execute simulations. The results are specialized to classical models as those by Vasicek, Hull and White and CIR with a semi-Markov modulation.
Using backward propagators, we construct inhomogeneous Random Evolutions on Banach spaces driven by (uniformly ergodic) Semi-Markov processes. After studying some of their properties (measurability, continuity, integral representation), we establish a Law of Large Numbers for such inhomogeneous Random Evolutions, and more precisely their weak convergence - in the Skorohod space $D$ - to an inhomogeneous semigroup. A martingale characterization of these inhomogeneous Random Evolutions is also obtained. Finally, we present applications to inhomogeneous Lévy Random Evolutions.
New algorithms for construction of asymptotic expansions for stationary distributions of nonlinearly perturbed semi-Markov processes with finite phase spaces are presented. These algorithms are based on a special technique of sequential phase space reduction, which can be applied to processes with an arbitrary asymptotic communicative structure of phase spaces. Asymptotic expansions are given in two forms, without and with explicit bounds for remainders.
We consider the problem of constructing an appropriate multivariate model for the study of the counterparty credit risk in credit rating migration problem. For this financial problem different multivariate Markov chain models were proposed. However the markovian assumption may be inappropriate for the study of the dynamic of credit ratings which typically show non markovian like behaviour. In this paper we develop a semi-Markov approach to the study of the counterparty credit risk by defining a new multivariate semi-Markov chain model. Methods are given for computing the transition probabiliti…
We investigate in a systematic way hypercontractivity property in Orlicz spaces for Markov semi-groups related to homogeneous and non homogeneous diffusions in $\mathbb{R}^{n}$. We provide an explicit construction of a family of Orlicz functions for which we prove that the associated hypercontractivity property is equivalent to a suitable functional inequality.
In this short note we consider semi-Markov processes satisfying the condition of direction-time independence (Markov renewal processes). We derive large deviation principles and fluctuation theorems for the empirical current and the empirical currents along cycles. Our derivation is based on the joint LDP for the empirical measure and flow recently proved in \cite{MZ}.
We consider sampling and enumeration problems for Markov equivalence classes. We create and analyze a Markov chain for uniform random sampling on the DAGs inside a Markov equivalence class. Though the worst case is exponentially slow mixing, we find a condition on the Markov equivalence class for polynomial time mixing. We also investigate the ratio of Markov equivalence classes to DAGs and a Markov chain of He, Jia, and Yu for random sampling of sparse Markov equivalence classes.
In this paper, we study Markov dynamics on unitary duals of compact quantum groups. We construct such dynamics from characters of quantum groups. Then we show that the dynamics have generators, and we give an explicit formula of the generators using the representation theory. Moreover, we construct Markov dynamics on the unitary dual of an inductive limit of compact quantum groups.
Invariance properties of linear functionals and linear maps on algebras of functions on quantum homogeneous spaces are studied, in particular for the special case of expected coideal *-subalgebras. Several one-to-one correspondences between such invariant functionals are established. Adding a positivity condition, this yields one-to-one correspondences of invariant quantum Markov semigroups acting on expected coideal *-subalgebras and certain convolution semigroups of states on the underlying compact quantum group. This gives an approach to classifying invariant quantum Markov semigroups on th…
We study coercive inequalities in Orlicz spaces associated to the probability measures on finite and infinite dimensional spaces which tails decay slower than the Gaussian ones. We provide necessary and sufficient criteria for such inequalities to hold and discuss relations between various classes of inequalities.
This article settles the convergence question for multivariate barycentric subdivision schemes with nonnegative masks on complete metric spaces of nonpositive Alexandrov curvature, also known as Hadamard spaces. We establish a link between these types of refinement algorithms and the theory of Markov chains by characterizing barycentric subdivision schemes as nonlinear Markov semigroups. Exploiting this connection, we subsequently prove that any such scheme converges on arbitrary Hadamard spaces if and only if it converges for real valued input data. Moreover, we generalize a characterization …
We introduce generalized quantum Markov states and generalized d-Markov chains which extend the notion quantum Markov chains on spin systems to that on $C^*$-algebras defined by general graphs. As examples of generalized d-Markov chains, we construct the entangled Markov fields on tree graphs. The concrete examples of generalized d-Markov chains on Cayley trees are also investigated.
We use results from zero-error information theory to determine the set of non-injective functions through which a Markov chain can be projected without losing information. These lumping functions can be found by clique partitioning of a graph related to the Markov chain. Lossless lumping is made possible by exploiting the (sufficiently sparse) temporal structure of the Markov chain. Eliminating edges in the transition graph of the Markov chain trades the required output alphabet size versus information loss, for which we present bounds.
In this paper we introduce a new approach to discrete-time semi-Markov decision processes based on the sojourn time process. Different characterizations of discrete-time semi-Markov processes are exploited and decision processes are constructed by their means. With this new approach, the agent is allowed to consider different actions depending also on the sojourn time of the process in the current state. A numerical method based on $Q$-learning algorithms for finite horizon reinforcement learning and stochastic recursive relations is investigated. Finally, we consider two toy examples: one in …
This paper presents a novel theoretical Monte Carlo Markov chain procedure in the framework of graphs. It specifically deals with the construction of a Markov chain whose empirical distribution converges to a given reference one. The Markov chain is constrained over an underlying graph, so that states are viewed as vertices and the transition between two states can have positive probability only in presence of an edge connecting them. The analysis is carried out on the basis of the relationship between the support of the target distribution and the connectedness of the graph.
