考虑一个具有相当一般状态空间的离散时间马尔可夫链，其具有不变概率测度 $μ$。文献中存在若干充分条件，可保证所有或 $μ$-几乎所有的转移概率在总变差（TV）度量下收敛到 $μ$：不可约性加非周期性、转移概率的等价性质，或耦合性质。在本工作中，我们回顾并改进了其中一些判据，使其分别成为所有或 $μ$-几乎所有转移概率 TV 收敛的充分必要条件。此外，我们讨论了所谓的广义耦合。
英文摘要：
Consider a discrete time Markov chain with rather general state space which has an invariant probability measure $μ$. There are several sufficient conditions in the literature which guarantee convergence of all or $μ$-almost all transition probabilities to $μ$ in the total variation (TV) metric: irreducibility plus aperiodicity, equivalence properties of transition probabilities, or coupling properties. In this work, we review and improve some of these criteria in such a way that they become necessary and sufficient for TV convergence of all respectively $μ$-almost all transition probabilities. In addition, we discuss so-called generalized couplings.
We consider the problem of characterising expected hitting times and hitting probabilities for imprecise Markov chains. To this end, we consider three distinct ways in which imprecise Markov chains have been defined in the literature: as sets of homogeneous Markov chains, as sets of more general stochastic processes, and as game-theoretic probability models. Our first contribution is that all these different types of imprecise Markov chains have the same lower and upper expected hitting times, and similarly the hitting probabilities are the same for these three types. Moreover, we provide a ch…
Suppose that $\mathcal C$ is a finite collection of patterns. Observe a Markov chain until one of the patterns in $\mathcal C$ occurs as a run. This time is denoted by $τ$. In this paper, we aim to give an easy way to calculate the mean waiting time $E(τ)$ and the stopping probabilities $P(τ=τ_A)$ with $A\in\mathcal C$, where $τ_A$ is the waiting time until the pattern $A$ appears as a run.
We propose a discrete time discrete space Markov chain approximation with a Brownian bridge correction for computing curvilinear boundary crossing probabilities of a general diffusion process on a finite time interval. For broad classes of curvilinear boundaries and diffusion processes, we prove the convergence of the constructed approximations in the form of products of the respective substochastic matrices to the boundary crossing probabilities for the process as the time grid used to construct the Markov chains is getting finer. Numerical results indicate that the convergence rate for the p…
This paper develops asymptotics and approximations for ruin probabilities in a multivariate risk setting. We consider a model in which the individual reserve processes are driven by a common Markovian environmental process. We subsequently consider a regime in which the claim arrival intensity and transition rates of the environmental process are jointly sped up, and one in which there is (with overwhelming probability) maximally one transition of the environmental process in the time interval considered. The approximations are extensively tested in a series of numerical experiments.
We study Markov chains generated by iterated Lipschitz functions systems with possibly place dependent probabilities. Under general conditions, we prove uniqueness of the invariant probability measure for the associated Markov chain, by using quasi-compact linear operators technics. We use the same approach to describe the behavior of the Diaconis-Friedman's chain on [0,1] with possibly place dependent probabilities.
In this paper, we consider a class of inhomogeneous semi-Markov processes directly based on intensity processes for marked point processes. We show that this class satisfies the semi-Markov properties defined elsewhere in the literature. We use the marked point process setting to derive strong upper bounds on various probabilities for semi-Markov processes. Using these bounds, we rigorously prove for the case of countably infinite state space that the transition intensities are right-derivatives of the transition probabilities, and we prove for the case of finite state space that the transitio…
We consider two types of discrete-time Markov chains where the state space is a graded poset and the transitions are taken along the covering relations in the poset. The first type of Markov chain goes only in one direction, either up or down in the poset (an \emph{up chain} or \emph{down chain}). The second type toggles between two adjacent rank levels (an \emph{up-and-down chain}). We introduce two compatibility concepts between the up-directed transition probabilities (an \emph{up rule}) and the down-directed (a \emph{down rule}), and we relate these to compatibility between up-and-down cha…
In this paper, we study Ornstein-Uhlenbeck processes with Markov modulation, whose parameters depend on an external underlying two-state Markov process. Conditional mean and variance of such processes under given modulation are investigated from the point of view of the first passage probabilities and invariant measures. It is also studied the limiting behaviour under scaling conditions similar to Kac's scaling.
We consider perturbations of positive recurrent Markov modulated fluid models. In addition to the infinitesimal generator of the phases, we also perturb the rate matrix, and analyze the effect of those perturbations on the matrix of first return probabilities to the initial level. Our main contribution is the construction of a substitute for the matrix of first return probabilities, which enables us to analyze the effect of the perturbation under consideration.
The asymptotic variance is an important criterion to evaluate the performance of Markov chains, especially for the central limit theorems. We give the variational formulas for the asymptotic variance of discrete-time (non-reversible) Markov chains on general state space. The variational formulas provide many applications, extending the classical Peskun's comparison theorem to non-reversible Markov chains, and obtaining several comparison theorems between Markov chains with various perturbations.
In this paper we discuss weak convergence of continuous-time Markov chains to a non-symmetric pure jump process. We approach this problem using Dirichlet forms as well as semimartingales. As an application, we discuss how to approximate a given Markov process by Markov chains.
In this paper we continue the study of conditional Markov chains (CMCs) with finite state spaces, that we initiated in Bielecki, Jakubowski and Niewęgłowski (2015). Here, we turn our attention to the study of Markov consistency and Markov copulae with regard to CMCs, and thus we follow up on the study of Markov consistency and Markov copulae for ordinary Markov chains that we presented in Bielecki, Jakubowski and Niewęgłowski (2013).
We study inhomogeneous continuous-time weakly ergodic Markov chains with a finite state space. We introduce the notion of a Markov chain with the regular structure of an infinitesimal matrix and study the sharp upper bounds on the rate of convergence for such class of Markov chains.
In this paper, we present reversibility preserving operations on Markov chain transition matrices. Simple row and column operations allow us to create new reversible transition matrices and yield an easy method for checking a Markov chain for reversibility.
We define fully coupled forward-backward stochastic differential equations on spaces related to continuous time, finite state Markov Chains. Existence and uniqueness results of the fully coupled forward-backward stochastic differential equations on Markov Chains are obtained.
We prove that an irreducible aperiodic Markov chain is geometrically ergodic if and only if any separately bounded functional of the stationary chain satisfies an appropriate subgaussian deviation inequality from its mean.
Extensions of Kemeny's constant, as derived for irreducible finite Markov chains in discrete time, to Markov renewal processes and Markov chains in continuous time are discussed. Three alternative Kemeny's functions and their variants are considered. Typically, they lead to a constant if and only if the mean holding times between the states in the Markov renewal process are constant. However one particular variant leads to a constant, analogous to the discrete time Markov chain result. Specifically, if the state space is finite, the weighted sum of the mean first passage times (omitting the me…
We consider periodic Markov chains with absorption. Applying to iterates of this periodic Markov chain criteria for the exponential convergence of conditional distributions of aperiodic absorbed Markov chains, we obtain exponential estimates for the periodic asymptotic behavior of the semigroup of the Markov chain. This implies in particular the exponential convergence in total variation of the conditional distribution of the Markov chain given non-absorption to a periodic sequence of limit measures and we characterize the cases where this sequence is constant, which corresponds to the cases w…
We study an approximation method of stationary characters of a two-dimensional Markov chain via the Stein method. For this purpose, innovative methods are developed to estimate the moments of the Markov chain, as well as the solution to the Poisson equation with a partial differential operator.
Let $(ξ_n)_{n=0}^\infty$ be a nonhomogeneous Markov chain taking values from finite state-space of $\mathbf{X}=\{1,2,\ldots,b\}$. In this paper, we will study the generalized entropy ergodic theorem with almost-everywhere and $\mathcal{L}_1$ convergence for nonhomogeneous Markov chains, which generalizes the corresponding classical results for the Markov chains.
In this paper, we quantify some known approximation to the Curie-Weiss model via applying the Stein method to the Markov chain whose stationary distribution coincides with Curie-Weiss model.
We are interested in studying the sensitivity of diffusion processes or their approximations by Markov Chains with respect to a perturbation of the coefficients.
Starting from a Markov chain with a finite alphabet, we consider the chain obtained when all but one symbol are undistinguishable for the practitioner. We study necessary and sufficient conditions for this chain to have continuous transition probabilities with respect to the past.
Improved rates of convergence for ergodic Markov chains and relaxed conditions for them, as well as analogous convergence results for some non-homogeneous Markov chains are studied. The setting from the previous works is extended. Examples are provided where the new bounds are better and where they give the same convergence rate as in the classical Markov -- Dobrushin inequality.
This is the story of the encounter between two worlds: the world of random walks and the world of Variable Length Markov Chains (VLMC). The meeting point turns around the semi-Markov property of underlying processes.
In this paper we are concerned with hitting times of a family of density-dependent Markov chains. A moderate deviation principle of the hitting time is given. The proof of the main theorem relies heavily on moderate deviations of density-dependent Markov chains given in \cite{Xue2021} and upper bounds of large deviations of Markov processes given in \cite{Dupuis1991}. An analogue moderate deviation of the hitting time of the diffusion approximation of the density-dependent Markov chain introduced in \cite{Ethier1986} is also given.
We study certain properties of the function space of autocorrelation functions of Unit Continuous Time Markov Chains (CTMCs). It is shown that under particular conditions, the $L^p$ norm of the autocorrelation function of arbitrary finite state space CTMCs is infinite. Several interesting inferences are made for point processes associated with CTMCs/ Discrete Time Markov Chains (DTMCs).
In this paper we study the additive functionals of Markov chains via conditioning with respect to both past and future of the chain. We shall point out new sufficient projective conditions, which assure that the variance of partial sums of n consecutive random variables of a stationary Markov chain is linear in n. The paper also addresses the central limit theorem problem and is listing several open questions.
On conditions for Dobrushin's Central limit theorem for non-homogeneous Markov chains
A.A. Kharkevich Institute for Information Transmission Problems of Russian Academy of Sciences（俄罗斯科学院信息传输问题阿·阿·哈尔维奇研究所）；S.M. Nikol’skii Mathematical Institute, RUDN University（俄罗斯人民友谊大学尼古利斯基数学研究所）；National Research University Higher School of Economics（国立高等经济大学）
The present paper establishes a discrete version of the result obtained by P. Carr and S. Nadtochiy (2011) for 1-dimensional diffusion processes. Our result is for Markov chains on $\mathbf{Z}^d$.
This paper studies the last-column-block-augmented northwest-corner truncation (LC-block-augmented truncation, for short) of discrete-time block-monotone Markov chains under subgeometric drift conditions. The main result of this paper is to present an upper bound for the total variation distance between the stationary probability vectors of a block-monotone Markov chain and its LC-block-augmented truncation. The main result is extended to Markov chains that themselves may not be block monotone but are block-wise dominated by block-monotone Markov chains satisfying modified drift conditions. Fi…
The paper is devoted to studies of perturbed Markov chains commonly used for description of information networks. In such models, the matrix of transition probabilities for the corresponding Markov chain is usually regularised by adding a special damping matrix multiplied by a small damping (perturbation) parameter $\varepsilon$. We give effective upper bounds for the rate of approximation for stationary distributions of unperturbed Markov chains by stationary distributions of perturbed Markov chains with regularised matrices of transition probabilities, asymptotic expansions for approximating…
Motivated by the notion of resistance distance on graph, we define a new resistance distance between two states on a given finite ergodic Markov chain based on its fundamental matrix. We prove a few equivalent formulations and discuss its relation with other parameters of the Markov chain such as its group inverse, stationary distribution, eigenvalues or hitting time. In addition, building upon existing sum rules for the hitting time of Markov chain, we give sum rules of this new resistance distance of Markov chains that resembles the sum rules of the resistance distance on graph. This yields …
