Many natural populations are well modelled through time-inhomogeneous stochastic processes. Such processes have been analysed in the physical sciences using a method based on Lie algebras, but this methodology is not widely used for models with ecological, medical and social applications. This paper presents the Lie algebraic method, and applies it to three biologically well motivated examples. The result of this is a solution form that is often highly computationally advantageous.
In this paper we derive the Wiener-Hopf factorization for a finite-state time-inhomogeneous Markov chain. To the best of our knowledge, this study is the first attempt to investigate the Wiener-Hopf factorization for time-inhomogeneous Markov chains. In this work we only deal with a special class of time-inhomogeneous Markovian generators, namely piece-wise constant, which allows to use an appropriately tailored randomization technique. Besides the mathematical importance of the Wiener-Hopf factorization methodology, there is also an important computational aspect: it allows for efficient comp…
Using backward propagators, we construct inhomogeneous Random Evolutions on Banach spaces driven by (uniformly ergodic) Semi-Markov processes. After studying some of their properties (measurability, continuity, integral representation), we establish a Law of Large Numbers for such inhomogeneous Random Evolutions, and more precisely their weak convergence - in the Skorohod space $D$ - to an inhomogeneous semigroup. A martingale characterization of these inhomogeneous Random Evolutions is also obtained. Finally, we present applications to inhomogeneous Lévy Random Evolutions.
This work contributes to the theory of Wiener-Hopf type factorization for finite Markov chains. This theory originated in the seminal paper Barlow et al. (1980), which treated the case of finite time-homogeneous Markov chains. Since then, several works extended the results of Barlow et al. (1980) in many directions. However, all these extensions were dealing with time-homogeneous Markov case. The first work dealing with the time-inhomogeneous situation was Bielecki et al. (2018), where Wiener-Hopf type factorization for time-inhomogeneous finite Markov chain with piecewise constant generator m…
We provide an original and general sufficient criterion ensuring the exponential contraction of Feynman-Kac semi-groups of penalized processes. This criterion is applied to time-inhomogeneous one-dimensional diffusion processes conditioned not to hit 0 and to penalized birth and death processes evolving in a quenched random environment.
Comparison results are given for time-inhomogeneous Markov processes with respect to function classes induced stochastic orderings. The main result states comparison of two processes, provided that the comparability of their infinitesimal generators as well as an invariance property of one process is assumed. The corresponding proof is based on a representation result for the solutions of inhomogeneous evolution problems in Banach spaces, which extends previously known results from the literature. Based on this representation, an ordering result for Markov processes induced by bounded and unbo…
We study time-inhomogeneous Markov chains with finite state spaces using Nash and logarithmic-Sobolev inequalities, and the notion of $c$-stability. We develop the basic theory of such functional inequalities in the time-inhomogeneous context and provide illustrating examples.
We consider a time inhomogeneous strong Markov process $(ξ_t)_{t\ge 0}$ taking values in a Polish state space whose semigroup has a $T$-periodic structure. We give simple conditions which imply ergodicity of the grid chain $(ξ_{kT})_{k\in \mathbb{N}_0}$ In case of $d$-dimensional possibly degenerate diffusions, the conditions are stated in terms of drift and diffusion coefficient of the process.
We suggest an approach to obtaining general two-sided bounds on the rate of convergence in terms of special "weighted" norms related to total variation. Some important classes of continuous-time Markov chains are considered: birth-death-catastrophes processes, queueing models with batch arrivals and group services, chains with absorption in zero.
Starting from a given Markov kernel on a finite set $V$ and a bijection $g$ of $V$, we construct and study a time inhomogeneous Markov chain whose kernel at time $n$ is obtained from $K$ by transport of $g^{n-1}$. We show that this construction leads to interesting examples, and we obtain quantitative results for some of these examples.
We discuss problems posed by the quantitative study of time inhomogeneous Markov chains. The two main notions for our purpose are merging and stability. Merging (also called weak ergodicity) occurs when the chain asymptotically forgets where it started. It is a loss of memory property. Stability relates to the question of whether or not, despite temporary variations, there is a rough shape describing the long time behavior of the chain. For instance, we will discuss an example where the long time behavior is roughly described by a binomial, with temporal variations.
In this work, we consider an inhomogeneous (discrete time) Markov chain and are interested in its long time behavior. We provide sufficient conditions to ensure that some of its asymptotic properties can be related to the ones of a homogeneous (continuous time) Markov process. Renowned examples such as a bandit algorithms, weighted random walks or decreasing step Euler schemes are included in our framework. Our results are related to functional limit theorems, but the approach differs from the standard "Tightness/Identification" argument; our method is unified and based on the notion of pseudo…
In this paper, we consider a class of inhomogeneous semi-Markov processes directly based on intensity processes for marked point processes. We show that this class satisfies the semi-Markov properties defined elsewhere in the literature. We use the marked point process setting to derive strong upper bounds on various probabilities for semi-Markov processes. Using these bounds, we rigorously prove for the case of countably infinite state space that the transition intensities are right-derivatives of the transition probabilities, and we prove for the case of finite state space that the transitio…
In this paper, we consider two time-inhomogeneous Markov chains $X^{(l)}_t$, $l\in\{1,2\}$, with discrete time on a general state space. We assume the existence of some renewal set $C$ and investigate the time of simultaneous renewal, that is, the first positive time when the chains hit the set $C$ simultaneously. The initial distributions for both chains may be arbitrary. Under the condition of stochastic domination and nonlattice condition for both renewal processes, we derive an upper bound for the expectation of the simultaneous renewal time. Such a bound was calculated for two time-inhomo…
The main subject of the study in this paper is the simultaneous renewal time for two time-inhomogeneous Markov chains which start with arbitrary initial distributions. By a simultaneous renewal we mean the first time of joint hitting the specific set $C$ by both processes. Under the condition of existence a dominating sequence for both renewal sequences generated by the chains and non-lattice condition for renewal probabilities an upper bound for the expectation of the simultaneous renewal time is obtained.
This paper deals with ergodic theorems for particular time-inhomogeneous Markov processes, whose the time-inhomogeneity is asymptotically periodic. Under a Lyapunov/minorization condition, it is shown that, for any measurable bounded function $f$, the time average $\frac{1}{t} \int_0^t f(X_s)ds$ converges in $\mathbb{L}^2$ towards a limiting distribution, starting from any initial distribution for the process $(X_t)_{t \geq 0}$. This convergence can be improved to an almost sure convergence under an additional assumption on the initial measure. This result will be then applied to show the exis…
We obtain a representation of an inhomogeneous Levy process in a Lie group or a homogeneous space in terms of a drift, a matrix function and a measure function. Because the stochastic continuity is not assumed, our result generalizes the well known Levy-Ito representation for stochastic continuous processes with independent increments in Euclidean spaces and the extension to Lie groups.
Convergence rates of Markov chains have been widely studied in recent years. In particular, quantitative bounds on convergence rates have been studied in various forms by Meyn and Tweedie [Ann. Appl. Probab. 4 (1994) 981-1101], Rosenthal [J. Amer. Statist. Assoc. 90 (1995) 558-566], Roberts and Tweedie [Stochastic Process. Appl. 80 (1999) 211-229], Jones and Hobert [Statist. Sci. 16 (2001) 312-334] and Fort [Ph.D. thesis (2001) Univ. Paris VI]. In this paper, we extend a result of Rosenthal [J. Amer. Statist. Assoc. 90 (1995) 558-566] that concerns quantitative convergence rates for time-homog…
We investigate the dissipativity properties of a class of scalar second order parabolic partial differential equations with time-dependent coefficients. We provide explicit condition on the drift term which ensure that the relative entropy of one particular orbit with respect to some other one decreases to zero. The decay rate is obtained explicitly by the use of a Sobolev logarithmic inequality for the associated semigroup, which is derived by an adaptation of Bakry's $Γ-$ calculus. As a byproduct, the systematic method for constructing entropies which we propose here also yields the well-kno…
A multi--state life insurance model is naturally described in terms of the intensity matrix of an underlying (time--inhomogeneous) Markov process which describes the dynamics for the states of an insured person. Between and at transitions, benefits and premiums are paid, defining a payment process, and the technical reserve is defined as the present value of all future payments of the contract. Classical methods for finding the reserve and higher order moments involve the solution of certain differential equations (Thiele and Hattendorf, respectively). In this paper we present an alternative m…
This paper continues the discussion on the stability of time-inhomogeneous Markov chains. In particular, this paper defines a time-inhomogeneous, discrete-time Markov chain governed by a continuous evolution in the appropriate martrix space. This matrix space, $\mathcal{P}_{n}^{ia}$, is the space of all stochastic matrices that are irreducible and aperiodic. For this new type of evolution there is a definition of a specific type of stability called the stable adiabatic time. This measure is bounded by a function of the optimal mixing time over the evolution. Namely, for a time-inhomogeneous, d…
We prove existence and uniqueness of solutions to Fokker--Planck equations associated to Markov operators multiplicatively perturbed by degenerate time-inhomogeneous coefficients. Precise conditions on the time-inhomogeneous coefficients are given. In particular, we do not necessarily require the coefficients to be neither globally bounded nor bounded away from zero. The approach is based on constructing random time-changes and studying related martingale problems for Markov processes with values in locally compact, complete and separable metric spaces.
We prove a Berry-Esseen theorem and Edgeworth expansions for partial sums of the form $S_N=\sum_{n=1}^{N}f_n(X_n,X_{n+1})$, where $\{X_n\}$ is a uniformly elliptic inhomogeneous Markov chain and $\{f_n\}$ is a sequence of uniformly bounded functions. The Berry-Esseen theorem holds without additional assumptions, while expansions of order $1$ hold when $\{f_n\}$ is irreducible, which is an optimal condition. For higher order expansions, we then focus on two situations. The first is when the essential supremum of $f_n$ is of order $O(n^{-\be})$ for some $\be\in(0,1/2)$. In this case it turns out…
Any exchangeable Markov processes on $[k]^{\mathbb{N}}$ with cadlag sample paths projects to a Markov process on the simplex whose sample paths are cadlag and of locally bounded variation. Furthermore, any such process has a de Finetti-type description as a mixture of i.i.d. copies of time-inhomogeneous Markov processes on $[k]$. In the Feller case, these time-inhomogeneous Markov processes have a relatively simple structure; however, in the non-Feller case a greater variety of behaviors is possible since the transition law of the underlying Markov process on $[k]^{\zz{N}}$ can depend in a non…
In this paper, we demonstrate through the use of matrix calculus a transparent analysis of fractional inhomogeneous Markov models for life insurance where transition matrices commute. The resulting formulae are intuitive matrix generalizations of their single-state counterparts, and the absorption times are matrix versions of well-known scalar distributions. A further advantage of this approach is that it allows extending the analysis to the non-Markovian case where sojourns are Mittag-Leffler distributed, and where the absorption times are fractional phase-type distributed. Considering determ…
We review some developments concerning Markov and Feller processes with jumps in geometric settings. These include stochastic differential equations in Markus canonical form, the Courrège theorem on Lie groups, and invariant Markov processes on manifolds under both transitive and more general Lie group actions.
We are interested in the dynamic of a structured branching population where the trait of each individual moves according to a Markov process. The rate of division of each individual is a function of its trait and when a branching event occurs, the trait of a descendant at birth depends on the trait of the mother. We prove a law of large numbers for the empirical distribution of ancestral trajectories. It ensures that the empirical measure converges to the mean value of the spine which is a time-inhomogeneous Markov process describing the trait of a typical individual along its ancestral lineag…
We construct a four parameter (z, z', a, b) family of Markov dynamics that preserve the z-measures on the boundary of the branching graph for classical Lie groups of type B, C, D. Our guiding principle is the "method of intertwiners" used previously in [7] to construct Markov processes that preserve the zw-measures.
We study inhomogeneous continuous-time weakly ergodic Markov chains with a finite state space. We introduce the notion of a Markov chain with the regular structure of an infinitesimal matrix and study the sharp upper bounds on the rate of convergence for such class of Markov chains.
In this paper we study time-inhomogeneous versions of one-dimensional Stochastic Differential Equations (SDE) involving the Local Time of the unknown process on curves. After proving existence and uniqueness for these SDE under mild assumptions, we explore their link with Parabolic Differential Equations (PDE) with transmission conditions. We study the regularity of solutions of such PDE and ensure the validity of a Feynman-Kac representation formula. These results are then used to characterize the solutions of these SDE as time-inhomogeneous Markov Feller processes.
We state and prove a generalized adiabatic theorem for Markov chains and provide examples and applications related to Glauber dynamics of Ising model over Z^d/nZ^d. The theorems derived in this paper describe a type of adiabatic dynamics for l^1(R_+^n) norm preserving, time inhomogeneous Markov transformations, while quantum adiabatic theorems deal with l^2(C^n) norm preserving ones, i.e. gradually changing unitary dynamics in C^n.
We provide explicit expressions for the constants involved in the characterisation of ergodicity of sub-geometric Markov chains. The constants are determined in terms of those appearing in the assumed drift and one-step minorisation conditions. The result is fundamental for the study of some algorithms where uniform bounds for these constants are needed for a family of Markov kernels. Our result accommodates also some classes of inhomogeneous chains.
From the datum of an integer partition and a classical Lie algebra, one can define a Markov chain on an associated multiplicative graph. For each classical family A, C, B, D, we thus obtain a sequence of Markov chain which is indexed by the rank of the considered algebra. In this article we show that, for each type, the transition kernel of the Markov chain has a limit when the rank tends to infinity. Moreover, the limit kernel does not depend on the considered type.
We extend the Ruzhansky-Turunen theory of pseudo differential operators on compact Lie groups into a tool that can be used to investigate group-valued Markov processes in the spirit of the work in Euclidean spaces of N.Jacob and collaborators. Feller semigroups, their generators and resolvents are exhibited as pseudo-differential operators and the symbols of the operators forming the semigroup are expressed in terms of the Fourier transform of the transition kernel. The symbols are explicitly computed for some examples including the Feller processes associated to stochastic flows arising from …
We consider the connections among `clumped' residual allocation models (RAMs), a general class of stick-breaking processes including Dirichlet processes, and the occupation laws of certain discrete space time-inhomogeneous Markov chains related to simulated annealing and other applications. An intermediate structure is introduced in a given RAM, where proportions between successive indices in a list are added or clumped together to form another RAM. In particular, when the initial RAM is a Griffiths-Engen-McCloskey (GEM) sequence and the indices are given by the random times that an auxiliary …
In this paper we study non-interactive correlation distillation (NICD), a generalization of the study of noise sensitivity of boolean functions. We extend the model to NICD on trees. In this model there is a fixed undirected tree with players at some of the nodes. One node is given a uniformly random string and this string is distributed throughout the network, with the edges of the tree acting as independent binary symmetric channels. The goal of the players is to agree on a shared random bit without communicating. Our new contributions include the following: 1. In the case of a k-leaf star g…
This paper studies pricing derivatives in an age-dependent semi-Markov modulated market. We consider a financial market where the asset price dynamics follow a regime switching geometric Brownian motion model in which the coefficients depend on finitely many age-dependent semi-Markov processes. We further allow the volatility coefficient to depend on time explicitly. Under these market assumptions, we study locally risk minimizing pricing of a class of European options. It is shown that the price function can be obtained by solving a non-local B-S-M type PDE. We establish existence and uniquen…
The paper is largely of a review nature. It considers two main methods used to study stability and obtain appropriate quantitative estimates of perturbations of (inhomogeneous) Markov chains with continuous time and a finite or countable state space. An approach is described to the construction of perturbation estimates for the main five classes of such chains associated with queuing models. Several specific models are considered for which the limit characteristics and perturbation bounds for admissible "perturbed" processes are calculated.
In this paper pseudo-differential operators with negative definite symbols are used to construct time- and space-inhomogeneous Markov processes. This is achieved by using the Markov evolution system associated with the fundamental solution of the corresponding pseudo-differential evolution equation. Negative definite symbols are non-standard and differ significantly from the class of Hörmander type symbols. The novelty of this work is the derivation and the representation of the fundamental solution as a Feynman path integral. This implies that the transition function of the constructed Markov…
We present a theorem which elucidates the connection between self-duality of Markov processes and representation theory of Lie algebras. In particular, we identify sufficient conditions such that the intertwining function between two representations of a certain Lie algebra is the self-duality function of a (Markov) operator. In concrete terms, the two representations are associated to two operators in interwining relation. The self-dual operator, which arise from an appropriate symmetric linear combination of them, is the generator of a Markov process. The theorem is applied to a series of ex…
We study a time-inhomogeneous SDE in $\R^d$ driven by a cylindrical Lévy process with independent coordinates which may have different scaling properties. Such a structure of the driving noise makes it strongly spatially inhomogeneous and complicates the analysis of the model significantly. We prove that the weak solution to the SDE is uniquely defined, is Markov, and has the strong Feller property. The heat kernel of the process is presented as a combination of an explicit `principal part' and a `residual part', subject to certain $L^\infty(dx)\otimes L^1(dy)$ and $L^\infty(dx)\otimes L^\inft…
Queueing networks are notoriously difficult to analyze sans both Markovian and stationarity assumptions. Much of the theoretical contribution towards performance analysis of time-inhomogeneous single class queueing networks has focused on Markovian networks, with the recent exception of work in Liu and Whitt (2011) and Mandelbaum and Ramanan (2010). In this paper, we introduce transitory queueing networks as a model of inhomogeneous queueing networks, where a large, but finite, number of jobs arrive at queues in the network over a fixed time horizon. The queues offer FIFO service, and we assum…
In this paper we continue our work on adiabatic time of time-inhomogeneous Markov chains first introduced in Kovchegov (2010) and Bradford and Kovchegov (2011). Our study is an analog to the well-known Quantum Adiabatic (QA) theorem which characterizes the quantum adiabatic time for the evolution of a quantum system as a result of applying of a series of Hamilton operators, each is a linear combination of two given initial and final Hamilton operators, i.e. $\mathbf{H}(s) = (1-s)\mathbf{H_0} + s\mathbf{H_1}$. Informally, the quantum adiabatic time of a quantum system specifies the speed at whi…
In this work, we consider a finite-state inhomogeneous-time Markov chain whose probabilities of transition from one state to another tend to decrease over time. This can be seen as a cooling of the dynamics of an underlying Markov chain. We are interested in the long time behavior of the empirical measure of this freezing Markov chain. Some recent papers provide almost sure convergence and convergence in distribution in the case of the freezing speed $n^{-θ}$, with different limits depending on $θ<1,θ=1$ or $θ>1$. Using stochastic approximation techniques, we generalize these results for any f…
