The paper is concerned with the equilibrium distributions of continuous-time density dependent Markov processes on the integers. These distributions are known typically to be approximately normal, and the approximation error, as measured in Kolmogorov distance, is of the smallest order that is compatible with their having integer support. Here, an approximation in the much stronger total variation norm is established, without any loss in the asymptotic order of accuracy; the approximating distribution is a translated Poisson distribution having the same variance and (almost) the same mean. Our arguments are based on the Stein-Chen method and Dynkin's formula.
It is shown that the method of exchangeable pairs introduced by Stein [Approximate Computation of Expectations (1986) IMS, Hayward, CA] for normal approximation can effectively be used for translated Poisson approximation. Introducing an additional smoothness condition, one can obtain approximation results in total variation and also in a local limit metric. The result is applied, in particular, to the anti-voter model on finite graphs as analyzed by Rinott and Rotar [Ann. Appl. Probab. 7 (1997) 1080--1105], obtaining the same rate of convergence, but now for a stronger metric.
The paper is concerned with approximating the distribution of a sum W of n integer valued random variables Y_i, whose distributions depend on the state of an underlying Markov chain X. The approximation is in terms of a translated Poisson distribution, with mean and variance chosen to be close to those of W, and the error is measured with respect to the total variation norm. Error bounds comparable to those found for normal approximation with respect to the weaker Kolmogorov distance are established, provided that the distribution of the sum of the Y_i's between the successive visits of X to a…
The mathematics of the finite single server queue with Poisson input and semi-Markov service times($M/SM/1/b$) is similar to that used for $BMAP/G/1/b$ systems. This observation results in new analytical formulas for a queue size in the $M/SM/1/b$ system. Both stationary and the transient solutions are considered
Stochastically monotone Markov chains arise in many applied domains, especially in the setting of queues and storage systems. Poisson's equation is a key tool for analyzing additive functionals of such models, such as cumulative sums of waiting times or sums of rewards. In this paper, we show that when the reward function for such a Markov chain is monotone, the solution of Poisson's equation is monotone. This implies that the value function associated with infinite horizon average reward is monotone in the state when the reward is monotone.
In this paper, the weak convergence of additive functionals of processes with locally independent increments and with Markov switching in the scheme of Poisson approximation is proved. For the relative compactness, a method proposed by R. Liptser for semimartingales is used with a modification, where we apply a solution of a singular perturbation problem instead of an ergodic theorem.
In this paper, we are devoted to developing matrix-analytic methods for solving Poisson's equation for irreducible and positive recurrent discrete-time Markov chains (DTMCs). Two special solutions, including the deviation matrix D and the expected additive-type functional matrix K, will be considered. The results are applied to Markov chains of GI/G/1-type and MAP/G/1 queues with negative customers. Further extensions to continuous-time Markov chains (CTMCs) are also investigated.
Generalization of the Lorden's inequality is an excellent tool for obtaining strong upper bounds for the convergence rate for various complicated stochastic models. This paper demonstrates a method for obtaining such bounds for some generalization of the Markov modulated Poisson process (MMPP). The proposed method can be applied to reliability and queuing theory.
This paper studies new Lancaster characterizations of bivariate multivariate Poisson, negative binomial and normal distributions which have diagonal expansions in multivariate orthogonal polynomials. The characterizations extend classical Lancaster characterizations of bivariate 1-dimensional distributions. Multivariate Poisson-Charlier, Meixner and Hermite-Chebycheff orthogonal polynomials, used in the characterizations, are constructed from classical 1-dimensional orthogonal polynomials and multivariate Krawtchouk polynomials. New classes of transition functions of discrete and continuous ti…
We consider the almost semi-continuous processes defined on a finite Markov chain. The representation of the moment generating functions for the absolute maximum after achievement positive level and for the recovery time are obtained. Modified processes with two-step rate of negative jumps are investigated.
This paper considers the Poisson equation for general state-space Markov chains in continuous time. The main purpose of this paper is to present specific bounds for the solutions of the Poisson equation for general state-space Markov chains. The solutions of the Poisson equation are unique in the sense that they are expressed in terms of a certain probabilistically interpretable solution (called the {\it standard solution}). Thus, we establish some specific bounds for the standard solution under the $f$-modulated drift condition (which is a kind of Foster-Lyapunov-type condition) and some mode…
Poisson's equation has a lot of applications in various areas. Usually it is hard to derive the explicit expression of the solution of Poisson's equation for a Markov chain on an infinitely many state space. We will present a computational framework for the solution for both discrete-time Markov chains (DTMCs) and continuous-time Markov chains (CTMCs), by developing the technique of augmented truncation approximations. The convergence to the solution is investigated in terms of the assumption about the monotonicity of the first return times, and is further established for two types of truncati…
Let P := {X_i,i >= 1} be a stationary Poisson point process in R^d, {C_i,i >= 1} be a sequence of i.i.d. random sets in R^d, and {Y_i^t; t \geq 0, i >= 1} be i.i.d. {0,1}-valued continuous time stationary Markov chains. We define the Markov-Poisson-Boolean model C_t := {Y_i^t(X_i + C_i), i >= 1}. C_t represents the coverage process at time t. We first obtain limit laws for k-coverage of an area at an arbitrary instant. We then obtain the limit laws for the k-coverage seen by a particle as it moves along a one-dimensional path.
Non-uniform estimates are obtained for Poisson, compound Poisson, translated Poisson, negative binomial and binomial approximations to sums of of m-dependent integer-valued random variables. Estimates for Wasserstein metric also follow easily from our results. The results are then exemplified by the approximation of Poisson binomial distribution, 2-runs and $m$-dependent $(k_1,k_2)$-events.
The paper concerns the classical occupancy scheme with infinitely many boxes. We establish approximations to the distributions of the number of occupied boxes, and of the number of boxes containing exactly r balls, within the family of translated Poisson distributions. These are shown to be of ideal asymptotic order, with respect both to total variation distance and to the approximation of point probabilities. The proof is probabilistic, making use of a translated Poisson approximation theorem of Röllin (2005).
A discrete-time Markov chain can be transformed into a new Markov chain by looking at its states along iterations of an almost surely finite stopping time. By the optional stopping theorem, any bounded harmonic function with respect to the transition function of the original chain is harmonic with respect to the transition function of the transformed chain. The reverse inclusion is in general not true. Our main result provides a sufficient condition on the stopping time which guarantees that the space of bounded harmonic functions for the transformed chain embeds in the space of bounded harmon…
This paper studies the subgeometric convergence of the stationary distribution in taking the infinite-level limit of a finite-level M/G/1-type Markov chain, that is, in letting the upper boundary level go to infinity. This study is performed through the fundamental deviation matrix, which is a block-decomposition-friendly solution for the Poisson equation of the deviation matrix. The fundamental deviation matrix yields a difference formula for the respective stationary distributions of the finite-level chain and the corresponding infinite-level chain. The difference formula plays a crucial rol…
This paper uses the generator approach of Stein's method to analyze the gap between steady-state distributions of Markov chains and diffusion processes. Until now, the standard way to invoke Stein's method for this problem was to use the Poisson equation for the diffusion as a starting point. The main technical difficulty with this approach is obtaining bounds on the derivatives of the solution to the Poisson equation, also known as Stein factor bounds. In this paper we propose starting with the discrete Poisson equation of the Markov chain. An important step in our approach is extending the d…
These are lecture notes on the subject defined in the title. As such, they do not pretend to be really new, probably except for the only section about Poisson equations with potentials. Yet, the hope of the author is that they may serve as a bridge to the important area of Poisson equations "in the whole space" and with a parameter, the latter theme not being presented here. Why this area is so important was explained in many papers and books (see the references [12, 34, 35]): it provides one of the main tools in diffusion approximation in the area stochastic averaging. Hence, the aim of these…
For a class of excitatory spiking neuron models with delayed feedback fed with a Poisson stochastic process, it is proven that the stream of output interspike intervals cannot be presented as a Markov process of any order. Keywords: spiking neuron; Poisson stochastic process; probability density function; delayed feedback; non-Markov stochastic process
Changing time of simple continuous-time Markov counting processes by independent unit-rate Poisson processes results in Markov counting processes for which we provide closed-form transition rates via composition of trajectories and with which we construct novel, simpler infinitesimally over-dispersed processes.
In this paper, we present new Poisson-type deviation inequalities for continuous-time Markov chains whose Wasserstein curvature or $Γ$-curvature is bounded below. Although these two curvatures are equivalent for Brownian motion on Riemannian manifolds, they are not comparable in discrete settings and yield different deviation bounds. In the case of birth--death processes, we provide some conditions on the transition rates of the associated generator for such curvatures to be bounded below and we extend the deviation inequalities established [Ané, C. and Ledoux, M. On logarithmic Sobolev inequa…
A stochastic ordering approach is applied with Stein's method for approximation by the equilibrium distribution of a birth-death process. The usual stochastic order and the more general s-convex orders are discussed. Attention is focused on Poisson and translated Poisson approximation of a sum of dependent Bernoulli random variables, for example k-runs in i.i.d. Bernoulli trials. Other applications include approximation by polynomial birth--death distributions.
In the paper we consider a generalizations of the notion of Poisson process to the case when classical convolution is replaced by generalized convolution in the sense of K. Urbanik [16] following two classical definitions of Poisson process. First, for every generalized convolution $\diamond$ we define $\diamond$-generalized Poisson process type I as a Markov process with the $\diamond$-generalized Poisson distribution. Such processes have stationary independent increments in the sense of generalized convolution, but usually they do not live on $\mathbb{N}_0$. The $\diamond$-generalized Poisso…
In this paper, we employ Markov process theory to prove asymptotic results for a class of stochastic processes which arise as solutions of a stochastic evolution inclusion and are given by the representation formula \begin{align*} \mathbb{X}_{x}(t)=\sum \limits_{m=0}\limits^{\infty}T((t-α_{m})_{+})(x_{x,m})1\hspace{-0,9ex}1_{[α_{m},α_{m+1})}(t), \end{align*} where $(T(t))_{t \geq 0}$ is a (nonlinear) time-continuous, contractive semigroup acting on a separable Banach space $(V,||\cdot||_{V})$, $(α_{m})_{m \in \mathbb{N}}$ is the sequence of arrival times of a homogeneous Poisson process, $x$ i…
Representations of branching Markov processes and their measure-valued limits in terms of countable systems of particles are constructed for models with spatially varying birth and death rates. Each particle has a location and a "level," but unlike earlier constructions, the levels change with time. In fact, death of a particle occurs only when the level of the particle crosses a specified level $r$, or for the limiting models, hits infinity. For branching Markov processes, at each time $t$, conditioned on the state of the process, the levels are independent and uniformly distributed on $[0,r]…
We revisit functional central limit theorems for additive functionals of ergodic Markov diffusion processes. Translated in the language of partial differential equations of evolution, they appear as diffusion limits in the asymptotic analysis of Fokker-Planck type equations. We focus on the square integrable framework, and we provide tractable conditions on the infinitesimal generator, including degenerate or anomalously slow diffusions. We take advantage on recent developments in the study of the trend to the equilibrium of ergodic diffusions. We discuss examples and formulate open problems.
This paper uses the generator comparison approach of Stein's method to analyze the gap between steady-state distributions of Markov chains and diffusion processes. The "standard" generator comparison approach starts with the Poisson equation for the diffusion, and the main technical difficulty is to obtain bounds on the derivatives of the solution to the Poisson equation, also known as Stein factor bounds. In this paper we propose starting with the Poisson equation of the Markov chain; we term this the prelimit approach. Although one still needs Stein factor bounds, they now correspond to fini…
The solution of Poisson's equation plays a key role in constructing the martingale through which sums of Markov correlated random variables can be analyzed. In this paper, we study two different representations for the solution in countable state space, one based on regenerative structure and the other based on an infinite sum of expectations. We also consider integrability and related uniqueness issues associated with solutions to Poisson's equation, and provide verifiable Lyapunov conditions to support our theory. Our key results include a central limit theorem and law of the iterated logari…
Stein's (1972) method is a very general tool for assessing the quality of approximation of the distribution of a random element by another, often simpler, distribution. In applications of Stein's method, one needs to establish a Stein identity for the approximating distribution, solve the Stein equation and estimate the behaviour of the solutions in terms of the metrics under study. For some Stein equations, solutions with good properties are known; for others, this is not the case. Barbour and Xia (1999) introduced a perturbation method for Poisson approximation, in which Stein identities for…
We prove a complete class theorem that characterizes \emph{all} stationary time reversible Markov processes whose finite dimensional marginal distributions (of all orders) are infinitely divisible. Aside from two degenerate cases (iid and constant), in both discrete and continuous time every such process with full support is a branching process with Poisson or Negative Binomial marginal univariate distributions and a specific bivariate distribution at pairs of times. As a corollary, we prove that every nondegenerate stationary integer valued processes constructed by the Markov thinning process…
This paper presents a new view of household epidemic models, where the interaction between the households is of mean field type. We thus obtain in the limit of infinitely many households a nonlinear Markov process solution of a McKean - Vlasov type Poisson driven SDE, and a propagation of chaos result. We also define a basic reproduction number R0, and show that if R0>1, then the nonlinear Markov process has a unique non trivial ergodic invariant probability measure, whereas if R0<=1, it converges to 0 as t tends to infinity.
We study the analogue of Poisson ensembles of Markov loops ('loop soups') in the setting of one-dimensional diffusions. We give a detailed description of the corresponding intensity measure. The properties of this measure on loops lead us to an extension of Vervaat's bridge-to-excursion transformation that relates the bridges conditioned by their minimum and the excursions of all the diffusion we consider and not just the Brownian motion. Further we describe the Poisson point process of loops, their occupation fields and explain how to sample these Poisson ensembles of loops using two-dimensio…
The paper is concerned with the equilibrium distribution $Π_n$ of the $n$-th element in a sequence of continuous-time density dependent Markov processes on the integers. Under a $(2+\a)$-th moment condition on the jump distributions, we establish a bound of order $O(n^{-(\a+1)/2}\sqrt{\log n})$ on the difference between the point probabilities of $Π_n$ and those of a translated Poisson distribution with the same variance. Except for the factor $\sqrt{\log n}$, the result is as good as could be obtained in the simpler setting of sums of independent integer-valued random variables. Our arguments…
In this paper the class of mixed renewal processes (MRPs for short) with mixing parameter a random vector from \cite{lm6z3} (enlarging Huang's \cite{hu} original class) is replaced by the strictly more comprising class of all extended MRPs by adding a second mixing parameter. We prove under a mild assumption, that within this larger class the basic problem, whether every Markov process is a mixed Poisson process with a random variable as mixing parameter has a solution to the positive. This implies the equivalence of Markov processes, mixed Poisson processes, and processes with the multinomial…
We describe a new construction of a family of measures on a group with the same Poisson boundary. Our approach is based on applying Markov stopping times to an extension of the original random walk.
Stochastic models for performance analysis, optimization and control of queues hinge on a multitude of alternatives for input point processes. In case of bursty traffic, one very popular model is the \textit{Markov Modulated Poisson Process} (MMPP), however it is not the only option. Here, we introduce an alternative that we call \textit{Markovian transition counting process} (MTCP). The latter is a point process counting the number of transitions of a finite continuous-time Markov chain. For a given MTCP one can establish an MMPP with the same first and second moments of counts. In this paper…
In this paper we prove the Poisson Hypothesis for the limiting behavior of the large queueing systems in some simple ("mean-field") cases. We show in particular that the corresponding dynamical systems, defined by the non-linear Markov processes, have a line of fixed points which are global attractors. To do this we derive the corresponding non-linear integral equation and we explore its self-averaging properties. Our derivation relies on a solution of a combinatorial problem of rode placements.
We give an elementary construction of a time-invertible Markov process which is discrete except at one instance. The process is one of the quadratic harnesses studied in our previous papers and can be regarded as a random joint of two independent Poisson processes.
We consider local alignments without gaps of two independent Markov chains from a finite alphabet, and we derive sufficient conditions for the number of essentially different local alignments with a score exceeding a high threshold to be asymptotically Poisson distributed. From the Poisson approximation a Gumbel approximation of the maximal local alignment score is obtained. The results extend those obtained by Dembo, Karlin and Zeitouni [Ann. Probab. 22 (1994) 2022--2039] for independent sequences of i.i.d. variables.
We study an approximation method of stationary characters of a two-dimensional Markov chain via the Stein method. For this purpose, innovative methods are developed to estimate the moments of the Markov chain, as well as the solution to the Poisson equation with a partial differential operator.
We consider a Markov chain on the space of (countable) partitions of the interval [0,1], obtained first by size biased sampling twice (allowing repetitions) and then merging the parts (if the sampled parts are distinct) or splitting the part uniformly (if the same part was sampled twice). We prove a conjecture of Vershik stating that the Poisson-Dirichlet law with parameter theta=1 is the unique invariant distribution for this Markov chain. Our proof uses a combination of probabilistic, combinatoric, and representation-theoretic arguments.
In this paper we study the Poisson Hypothesis, which is a device to analyze approximately the behavior of large queueing networks. We prove it in some simple limiting cases. We show in particular that the corresponding dynamical system, defined by the non-linear Markov process, has a line of fixed points which are global attractors. To do this we derive the corresponding non-linear equation and we explore its self-averaging properties. We also argue that in cases of havy-tail service times the PH can be violated.
In this paper we prove two results. First we show that dynamical systems with a $ϕ$-mixing measure have in the limit Poisson distributed return times almost everywhere. We use the Chen-Stein method to also obtain rates of convergence. Our theorem improves on previous results by allowing for infinite partitions and dropping the requirement that the invariant measure have finite entropy with respect to the given partition. As has been shown elsewhere, the limiting distribution at periodic points is not Poissonian (but compound Poissonian). Here we show that for all non-periodic points the return…
Let W be the number of points in (0,t] of a stationary finite-state Markov renewal point process. We derive a bound for the total variation distance between the distribution of W and a compound Poisson distribution. For any nonnegative random variable ζ, we construct a ``strong memoryless time'' \hat ζsuch that ζ-t is exponentially distributed conditional on {\hat ζ\leq t, ζ>t}, for each t. This is used to embed the Markov renewal point process into another such process whose state space contains a frequently observed state which represents loss of memory in the original process. We then write…
We study a two parameter family of processes with linear regressions and linear conditional variances. We give conditions for the unique solution of this problem, and point out the connection between the resulting Markov processes and the generalized convolutions introduced by Bożejko and Speicher.
Koenigs functions in the subcritical and critical Markov branching processes with Poisson probability reproduction of particles
Institute of Mathematics and Informatics, Bulgarian Academy of Sciences（保加利亚科学院数学与信息学研究所）；Centre of Excellence in Informatics and Information and Communication Technologies（信息与信息技术卓越中心）
