Convergence rates of Markov chains have been widely studied in recent years. In particular, quantitative bounds on convergence rates have been studied in various forms by Meyn and Tweedie [Ann. Appl. Probab. 4 (1994) 981-1101], Rosenthal [J. Amer. Statist. Assoc. 90 (1995) 558-566], Roberts and Tweedie [Stochastic Process. Appl. 80 (1999) 211-229], Jones and Hobert [Statist. Sci. 16 (2001) 312-334] and Fort [Ph.D. thesis (2001) Univ. Paris VI]. In this paper, we extend a result of Rosenthal [J. Amer. Statist. Assoc. 90 (1995) 558-566] that concerns quantitative convergence rates for time-homogeneous Markov chains. Our extension allows us to consider f-total variation distance (instead of total variation) and time-inhomogeneous Markov chains. We apply our results to simulated annealing.
In this paper we derive the Wiener-Hopf factorization for a finite-state time-inhomogeneous Markov chain. To the best of our knowledge, this study is the first attempt to investigate the Wiener-Hopf factorization for time-inhomogeneous Markov chains. In this work we only deal with a special class of time-inhomogeneous Markovian generators, namely piece-wise constant, which allows to use an appropriately tailored randomization technique. Besides the mathematical importance of the Wiener-Hopf factorization methodology, there is also an important computational aspect: it allows for efficient comp…
Using backward propagators, we construct inhomogeneous Random Evolutions on Banach spaces driven by (uniformly ergodic) Semi-Markov processes. After studying some of their properties (measurability, continuity, integral representation), we establish a Law of Large Numbers for such inhomogeneous Random Evolutions, and more precisely their weak convergence - in the Skorohod space $D$ - to an inhomogeneous semigroup. A martingale characterization of these inhomogeneous Random Evolutions is also obtained. Finally, we present applications to inhomogeneous Lévy Random Evolutions.
This work contributes to the theory of Wiener-Hopf type factorization for finite Markov chains. This theory originated in the seminal paper Barlow et al. (1980), which treated the case of finite time-homogeneous Markov chains. Since then, several works extended the results of Barlow et al. (1980) in many directions. However, all these extensions were dealing with time-homogeneous Markov case. The first work dealing with the time-inhomogeneous situation was Bielecki et al. (2018), where Wiener-Hopf type factorization for time-inhomogeneous finite Markov chain with piecewise constant generator m…
We provide an original and general sufficient criterion ensuring the exponential contraction of Feynman-Kac semi-groups of penalized processes. This criterion is applied to time-inhomogeneous one-dimensional diffusion processes conditioned not to hit 0 and to penalized birth and death processes evolving in a quenched random environment.
Comparison results are given for time-inhomogeneous Markov processes with respect to function classes induced stochastic orderings. The main result states comparison of two processes, provided that the comparability of their infinitesimal generators as well as an invariance property of one process is assumed. The corresponding proof is based on a representation result for the solutions of inhomogeneous evolution problems in Banach spaces, which extends previously known results from the literature. Based on this representation, an ordering result for Markov processes induced by bounded and unbo…
We study time-inhomogeneous Markov chains with finite state spaces using Nash and logarithmic-Sobolev inequalities, and the notion of $c$-stability. We develop the basic theory of such functional inequalities in the time-inhomogeneous context and provide illustrating examples.
We consider a time inhomogeneous strong Markov process $(ξ_t)_{t\ge 0}$ taking values in a Polish state space whose semigroup has a $T$-periodic structure. We give simple conditions which imply ergodicity of the grid chain $(ξ_{kT})_{k\in \mathbb{N}_0}$ In case of $d$-dimensional possibly degenerate diffusions, the conditions are stated in terms of drift and diffusion coefficient of the process.
We suggest an approach to obtaining general two-sided bounds on the rate of convergence in terms of special "weighted" norms related to total variation. Some important classes of continuous-time Markov chains are considered: birth-death-catastrophes processes, queueing models with batch arrivals and group services, chains with absorption in zero.
Starting from a given Markov kernel on a finite set $V$ and a bijection $g$ of $V$, we construct and study a time inhomogeneous Markov chain whose kernel at time $n$ is obtained from $K$ by transport of $g^{n-1}$. We show that this construction leads to interesting examples, and we obtain quantitative results for some of these examples.
We discuss problems posed by the quantitative study of time inhomogeneous Markov chains. The two main notions for our purpose are merging and stability. Merging (also called weak ergodicity) occurs when the chain asymptotically forgets where it started. It is a loss of memory property. Stability relates to the question of whether or not, despite temporary variations, there is a rough shape describing the long time behavior of the chain. For instance, we will discuss an example where the long time behavior is roughly described by a binomial, with temporal variations.
In this work, we consider an inhomogeneous (discrete time) Markov chain and are interested in its long time behavior. We provide sufficient conditions to ensure that some of its asymptotic properties can be related to the ones of a homogeneous (continuous time) Markov process. Renowned examples such as a bandit algorithms, weighted random walks or decreasing step Euler schemes are included in our framework. Our results are related to functional limit theorems, but the approach differs from the standard "Tightness/Identification" argument; our method is unified and based on the notion of pseudo…
In this paper, we consider a class of inhomogeneous semi-Markov processes directly based on intensity processes for marked point processes. We show that this class satisfies the semi-Markov properties defined elsewhere in the literature. We use the marked point process setting to derive strong upper bounds on various probabilities for semi-Markov processes. Using these bounds, we rigorously prove for the case of countably infinite state space that the transition intensities are right-derivatives of the transition probabilities, and we prove for the case of finite state space that the transitio…
In this paper, we consider two time-inhomogeneous Markov chains $X^{(l)}_t$, $l\in\{1,2\}$, with discrete time on a general state space. We assume the existence of some renewal set $C$ and investigate the time of simultaneous renewal, that is, the first positive time when the chains hit the set $C$ simultaneously. The initial distributions for both chains may be arbitrary. Under the condition of stochastic domination and nonlattice condition for both renewal processes, we derive an upper bound for the expectation of the simultaneous renewal time. Such a bound was calculated for two time-inhomo…
The main subject of the study in this paper is the simultaneous renewal time for two time-inhomogeneous Markov chains which start with arbitrary initial distributions. By a simultaneous renewal we mean the first time of joint hitting the specific set $C$ by both processes. Under the condition of existence a dominating sequence for both renewal sequences generated by the chains and non-lattice condition for renewal probabilities an upper bound for the expectation of the simultaneous renewal time is obtained.
This paper deals with ergodic theorems for particular time-inhomogeneous Markov processes, whose the time-inhomogeneity is asymptotically periodic. Under a Lyapunov/minorization condition, it is shown that, for any measurable bounded function $f$, the time average $\frac{1}{t} \int_0^t f(X_s)ds$ converges in $\mathbb{L}^2$ towards a limiting distribution, starting from any initial distribution for the process $(X_t)_{t \geq 0}$. This convergence can be improved to an almost sure convergence under an additional assumption on the initial measure. This result will be then applied to show the exis…
We investigate the dissipativity properties of a class of scalar second order parabolic partial differential equations with time-dependent coefficients. We provide explicit condition on the drift term which ensure that the relative entropy of one particular orbit with respect to some other one decreases to zero. The decay rate is obtained explicitly by the use of a Sobolev logarithmic inequality for the associated semigroup, which is derived by an adaptation of Bakry's $Γ-$ calculus. As a byproduct, the systematic method for constructing entropies which we propose here also yields the well-kno…
A multi--state life insurance model is naturally described in terms of the intensity matrix of an underlying (time--inhomogeneous) Markov process which describes the dynamics for the states of an insured person. Between and at transitions, benefits and premiums are paid, defining a payment process, and the technical reserve is defined as the present value of all future payments of the contract. Classical methods for finding the reserve and higher order moments involve the solution of certain differential equations (Thiele and Hattendorf, respectively). In this paper we present an alternative m…
This paper continues the discussion on the stability of time-inhomogeneous Markov chains. In particular, this paper defines a time-inhomogeneous, discrete-time Markov chain governed by a continuous evolution in the appropriate martrix space. This matrix space, $\mathcal{P}_{n}^{ia}$, is the space of all stochastic matrices that are irreducible and aperiodic. For this new type of evolution there is a definition of a specific type of stability called the stable adiabatic time. This measure is bounded by a function of the optimal mixing time over the evolution. Namely, for a time-inhomogeneous, d…
Many natural populations are well modelled through time-inhomogeneous stochastic processes. Such processes have been analysed in the physical sciences using a method based on Lie algebras, but this methodology is not widely used for models with ecological, medical and social applications. This paper presents the Lie algebraic method, and applies it to three biologically well motivated examples. The result of this is a solution form that is often highly computationally advantageous.
We prove existence and uniqueness of solutions to Fokker--Planck equations associated to Markov operators multiplicatively perturbed by degenerate time-inhomogeneous coefficients. Precise conditions on the time-inhomogeneous coefficients are given. In particular, we do not necessarily require the coefficients to be neither globally bounded nor bounded away from zero. The approach is based on constructing random time-changes and studying related martingale problems for Markov processes with values in locally compact, complete and separable metric spaces.
We prove a Berry-Esseen theorem and Edgeworth expansions for partial sums of the form $S_N=\sum_{n=1}^{N}f_n(X_n,X_{n+1})$, where $\{X_n\}$ is a uniformly elliptic inhomogeneous Markov chain and $\{f_n\}$ is a sequence of uniformly bounded functions. The Berry-Esseen theorem holds without additional assumptions, while expansions of order $1$ hold when $\{f_n\}$ is irreducible, which is an optimal condition. For higher order expansions, we then focus on two situations. The first is when the essential supremum of $f_n$ is of order $O(n^{-\be})$ for some $\be\in(0,1/2)$. In this case it turns out…
In this paper, we demonstrate through the use of matrix calculus a transparent analysis of fractional inhomogeneous Markov models for life insurance where transition matrices commute. The resulting formulae are intuitive matrix generalizations of their single-state counterparts, and the absorption times are matrix versions of well-known scalar distributions. A further advantage of this approach is that it allows extending the analysis to the non-Markovian case where sojourns are Mittag-Leffler distributed, and where the absorption times are fractional phase-type distributed. Considering determ…
We study inhomogeneous continuous-time weakly ergodic Markov chains with a finite state space. We introduce the notion of a Markov chain with the regular structure of an infinitesimal matrix and study the sharp upper bounds on the rate of convergence for such class of Markov chains.
We provide explicit expressions for the constants involved in the characterisation of ergodicity of sub-geometric Markov chains. The constants are determined in terms of those appearing in the assumed drift and one-step minorisation conditions. The result is fundamental for the study of some algorithms where uniform bounds for these constants are needed for a family of Markov kernels. Our result accommodates also some classes of inhomogeneous chains.
We investigate the problem of quantifying contraction coefficients of Markov transition kernels in Kantorovich ($L^1$ Wasserstein) distances. For diffusion processes, relatively precise quantitative bounds on contraction rates have recently been derived by combining appropriate couplings with carefully designed Kantorovich distances. In this paper, we partially carry over this approach from diffusions to Markov chains. We derive quantitative lower bounds on contraction rates for Markov chains on general state spaces that are powerful if the dynamics is dominated by small local moves. For Marko…
In this paper we study time-inhomogeneous versions of one-dimensional Stochastic Differential Equations (SDE) involving the Local Time of the unknown process on curves. After proving existence and uniqueness for these SDE under mild assumptions, we explore their link with Parabolic Differential Equations (PDE) with transmission conditions. We study the regularity of solutions of such PDE and ensure the validity of a Feynman-Kac representation formula. These results are then used to characterize the solutions of these SDE as time-inhomogeneous Markov Feller processes.
We study approximations of evolving probability measures by an interacting particle system. The particle system dynamics is a combination of independent Markov chain moves and importance sampling/resampling steps. Under global regularity conditions, we derive non-asymptotic error bounds for the particle system approximation. In a few simple examples, including high dimensional product measures, bounds with explicit constants of feasible size are obtained. Our main motivation are applications to sequential MCMC methods for Monte Carlo integral estimation.
We state and prove a generalized adiabatic theorem for Markov chains and provide examples and applications related to Glauber dynamics of Ising model over Z^d/nZ^d. The theorems derived in this paper describe a type of adiabatic dynamics for l^1(R_+^n) norm preserving, time inhomogeneous Markov transformations, while quantum adiabatic theorems deal with l^2(C^n) norm preserving ones, i.e. gradually changing unitary dynamics in C^n.
In this paper we investigate the behavior of the bridges of a Markov counting process in several directions. We first characterize convexity(concavity) in time of the mean value in terms of lower (upper) bounds on the so called \textit{reciprocal characteristics}. This result gives a natural criterion to determine whether bridges are "lazy" or "hurried". Under the hypothesis of global bounds on the reciprocal characteristics we prove sharp estimates for the marginal distributions and a comparison theorem for the jump times. When the height of the bridge tends to infinity we show the convergenc…
Quantitative Asymptotics for Time-Inhomogeneous Lévy-Driven SDEs with Asymptotically Vanishing Drifts
Center for Applied Mathematics, Tianjin University（天津大学应用数学中心）；School of Mathematics and Statistics & Key Laboratory of Analytical Mathematics and Applications (Ministry of Education) & Fujian Provincial Key Laboratory of Statistics and Artificial Intelligence, Fujian Normal University（福建师范大学数学与统计学院与分析数学及应用教育部重点实验室及福建省统计与人工智能省级重点实验室）
The paper is largely of a review nature. It considers two main methods used to study stability and obtain appropriate quantitative estimates of perturbations of (inhomogeneous) Markov chains with continuous time and a finite or countable state space. An approach is described to the construction of perturbation estimates for the main five classes of such chains associated with queuing models. Several specific models are considered for which the limit characteristics and perturbation bounds for admissible "perturbed" processes are calculated.
We study a time-inhomogeneous SDE in $\R^d$ driven by a cylindrical Lévy process with independent coordinates which may have different scaling properties. Such a structure of the driving noise makes it strongly spatially inhomogeneous and complicates the analysis of the model significantly. We prove that the weak solution to the SDE is uniquely defined, is Markov, and has the strong Feller property. The heat kernel of the process is presented as a combination of an explicit `principal part' and a `residual part', subject to certain $L^\infty(dx)\otimes L^1(dy)$ and $L^\infty(dx)\otimes L^\inft…
Queueing networks are notoriously difficult to analyze sans both Markovian and stationarity assumptions. Much of the theoretical contribution towards performance analysis of time-inhomogeneous single class queueing networks has focused on Markovian networks, with the recent exception of work in Liu and Whitt (2011) and Mandelbaum and Ramanan (2010). In this paper, we introduce transitory queueing networks as a model of inhomogeneous queueing networks, where a large, but finite, number of jobs arrive at queues in the network over a fixed time horizon. The queues offer FIFO service, and we assum…
In this paper one presents method for the computation of convergence bounds for four classes of multiserver queueing systems, described by inhomogeneous Markov chains. Specifically one considers inhomogeneous $M/M/S$ queueing system with possibly state-dependent arrival and service intensities and additionally possible batch arrivals and batch service. The unified approach based on logarithmic norm of linear operators for obtaining sharp upper and lower bounds on the rate of convergence and corresponding sharp perturbation bounds is described. As a side result, one shows by virtue of numerical…
In this paper we continue our work on adiabatic time of time-inhomogeneous Markov chains first introduced in Kovchegov (2010) and Bradford and Kovchegov (2011). Our study is an analog to the well-known Quantum Adiabatic (QA) theorem which characterizes the quantum adiabatic time for the evolution of a quantum system as a result of applying of a series of Hamilton operators, each is a linear combination of two given initial and final Hamilton operators, i.e. $\mathbf{H}(s) = (1-s)\mathbf{H_0} + s\mathbf{H_1}$. Informally, the quantum adiabatic time of a quantum system specifies the speed at whi…
In this work, we consider a finite-state inhomogeneous-time Markov chain whose probabilities of transition from one state to another tend to decrease over time. This can be seen as a cooling of the dynamics of an underlying Markov chain. We are interested in the long time behavior of the empirical measure of this freezing Markov chain. Some recent papers provide almost sure convergence and convergence in distribution in the case of the freezing speed $n^{-θ}$, with different limits depending on $θ<1,θ=1$ or $θ>1$. Using stochastic approximation techniques, we generalize these results for any f…
We treat the class of universal Markov processes on the d-dimensional Euklidean space which do not depend on random. For these, as well as for several subclasses, we prove criteria whether a function f, defined on the positive half-line, can be a path of a process in the respective class. This is useful in particular in the construction of (counter-)examples. Furthermore we characterize the processes of this kind, which are homogeneous in space and time. The semimartingale property is characterized in terms of the jumps of a one-dimensional deterministic Markov process. We emphasize the differ…
In this paper we study non-interactive correlation distillation (NICD), a generalization of the study of noise sensitivity of boolean functions. We extend the model to NICD on trees. In this model there is a fixed undirected tree with players at some of the nodes. One node is given a uniformly random string and this string is distributed throughout the network, with the edges of the tree acting as independent binary symmetric channels. The goal of the players is to agree on a shared random bit without communicating. Our new contributions include the following: 1. In the case of a k-leaf star g…
Distributional properties -including Laplace transforms- of integrals of Markov processes received a lot of attention in the literature. In this paper, we complete existing results in several ways. First, we provide the analytical solution to the most general form of Gaussian processes (with non-stationary increments) solving a stochastic differential equation. We further derive the characteristic function of integrals of Lévy-processes and Lévy driven Ornstein-Uhlenbeck processes with time-inhomogeneous coefficients based on the characteristic exponent of the corresponding stochastic integral…
