The probability that a transient Markov chain, or a Brownian path, will ever visit a given set Lambda, is classically estimated using the capacity of Lambda with respect to the Green kernel G(x,y). We show that replacing the Green kernel by the Martin kernel G(x,y)/G(0,y) yields improved estimates, which are exact up to a factor of 2. These estimates are applied to random walks on lattices, and also to explain a connection found by R. Lyons between capacity and percolation on trees.
In 1956, Dobrushin proved a definitive central limit theorem for non-homogeneous Markov chains. In this note, a shorter and different proof elucidating more the assumptions is given through martingale approximation.
We construct a nonstandard martingale from a discrete Markov chain. This is shown to be useful for solving the heat equation with a non smooth initial condition. We show that the nonstandard solution to the heat equation with a smooth initial condition specialises to the classical solution.
We present sufficient conditions, in terms of the jumping kernels, for two large classes of conservative Markov processes of pure-jump type to be purely discontinuous martingales with finite second moment. As an application, we establish the law of the iterated logarithm for sample paths of the associated processes.
Substitution Markov chains have been introduced [7] as a new model to describe molecular evolution. In this note, we study the associated Martin boundaries from a probabilistic and topological viewpoint. An example is given that, although having a boundary homeomorphic to the well-known coin tossing process, has a metric description that differs significantly.
We prove existence of boundary limits of ratios of positive harmonic functions for a wide class of Markov processes with jumps and irregular domains, in the context of general metric measure spaces. As a corollary, we prove uniqueness of the Martin kernel at each boundary point, that is, we identify the Martin boundary with the topological boundary. We also prove a Martin representation theorem for harmonic functions. Examples covered by our results include: strictly stable Lévy processes in R^d with positive continuous density of the Lévy measure; stable-like processes in R^d and in domains; …
It is well-known that well-posedness of a martingale problem in the class of continuous (or r.c.l.l.) solutions enables one to construct the associated transition probability functions. We extend this result to the case when the martingale problem is well-posed in the class of solutions which are continuous in probability. This extension is used to improve on a criterion for a probability measure to be invariant for the semigroup associated with the Markov process. We also give examples of martingale problems that are well-posed in the class of solutions which are continuous in probability but…
In our monograph with B. Roynette and M. Yor, we construct a sigma-finite measure related to penalisations of different stochastic processes, including the Brownian motion in dimension 1 or 2, and a large class of linear diffusions. In the last chapter of the monograph, we define similar measures from recurrent Markov chains satisfying some technical conditions. In the present paper, we give a classification of these measures, in function of the minimal Martin boundary of the Markov chain considered at the beginning. We apply this classification to the examples considered at the end of our mon…
We prove a new inequality controlling the large deviations of the empirical measure of a Markov chain. This inequality is based on the martingale used by Donsker and Varadhan and the minimax theorem. It holds for convex sets and it requires to take an infimum over the starting point. In the case of a compact space, this inequality is a partial improvement of the large deviations estimates of Donsker and Varadhan. In the case of a non compact space, we condition on the event that the process visits $n$ times a compact subset of the space and we still obtain a control on the exponential scale.
In 2012 Lau and Ngai, motivated by the work of Denker and Sato, gave an example of an isotropic Markov chain on the set of finite words over a three letter alphabet, whose Martin boundary is homeomorphic to the Sierpiński gasket. Here, we extend the results of Lau and Ngai to a class of non-isotropic Markov chains. We determine the Martin boundary and show that the minimal Martin boundary is a proper subset of the Martin boundary. In addition, we give a description of the set of harmonic functions.
The purpose of this article is to describe all possible beliefs of market participants on objective measures under Markovian environments when a risk-neutral measure is given. To achieve this, we employ the Martin integral representation of Markovian pricing kernels. Then, we offer economic and financial implications of this representation. This representation is useful to analyze the long-term behavior of the state variable in the market. The Ross recovery theorem and the long-term behavior of cash flows are discussed as applications.
The density-dependent Markov chain (DDMC) introduced in \cite{Kurtz1978} is a continuous time Markov process applied in fields such as epidemics, chemical reactions and so on. In this paper, we give moderate deviation principles of paths of DDMC under some generally satisfied assumptions. The proofs for the lower and upper bounds of our main result utilize an exponential martingale and a generalized version of Girsanov's theorem. The exponential martingale is defined according to the generator of DDMC.
We give conditions for the existence of a Yaglom limit for R-transient Markov chains with non-trivial rho-Martin entrance boundary (rho=1/R) and we characterize the rho-invariant limiting quasistationary distribution.
Consider an irreducible Markov chain which satisfies a ratio limit theorem, and let $ρ$ be the spectral radius of the chain. We investigate the relation of the the $ρ\,$-Martin boundary with the boundary induced by the $ρ\,$-harmonic kernel which appears in the ratio limit. Special emphasis is on random walks on non-amenable groups, specifically, free groups and hyperbolic groups.
Let $(\mathbb{P}^{s,x})_{(s,x)\in[0,T]\times E}$ be a family of probability measures, where $E$ is a Polish space,defined on the canonical probability space ${\mathbb D}([0,T],E)$ of $E$-valued cadlag functions. We suppose that a martingale problem with respect to a time-inhomogeneous generator $a$ is well-posed. We consider also an associated semilinear {\it Pseudo-PDE} with generator $a$ for which we introduce a notion of so called {\it decoupled mild} solution and study the equivalence with the notion of martingale solution introduced in a companion paper. We also investigate well-posedness…
We presented in \cite{bl2,bl7} an approach to derive the metastable behavior of continuous-time Markov chains. We assumed in these articles that the Markov chains visit points in the time scale in which it jumps among the metastable sets. We replace this condition here by assumtpions on the mixing times and on the relaxation times of the chains reflected at the boundary of the metastable sets.
In this paper, we prove Strassen's strong invariance principle for a vector-valued additive functionals of a Markov chain via the martingale argument and the theory of fractional coboundaries. The hypothesis is a moment bound on the resolvent.
In this paper we study the almost sure central limit theorem started from a point for additive functionals of a stationary and ergodic Markov chain via a martingale approximation in the almost sure sense. As a consequence we derive the quenched CLT for additive functionals or reversible Markov chains.
In the paper, the law of the iterated logarithm for additive functionals of Markov chains is obtained under some weak conditions, which are weaker than the conditions of invariance principle of additive functionals of Markov chains in M. Maxwell and M. Woodroofe (2000). The main technique is the martingale argument and the theory of fractional coboundaries.
The paper studies the time-homogeneous two-state Markov chains; the states are assumed to be binary symbols 0 and 1. The higher-order absolute differences taken from progressive states of a given chain are considered. A discrete capacity of subsets of natural series is defined and a limiting theorem for these differences, formulated in terms of Wiener criterion type relation, is presented.
We review recent results on the metastable behavior of continuous-time Markov chains derived through the characterization of Markov chains as unique solutions of martingale problems.
We develop a martingale approximation approach to studying the limiting behavior of quadratic forms of Markov chains. We use the technique to examine the asymptotic behavior of lag-window estimators in time series and we apply the results to Markov Chain Monte Carlo simulation. As another illustration, we use the method to derive a central limit theorem for U-statistics with varying kernels.
We derive sufficient and necessary optimality conditions in terms of a stochastic maximum principle (SMP) for controls associated with cost functionals of mean-field type, under dynamics driven by a class of Markov chains of mean-field type which are pure jump processes obtained as solutions of a well-posed martingale problem. As an illustration, we apply the result to generic examples of control problems as well as some applications.
In this paper we obtain the central limit theorem for triangular arrays of non-homogeneous Markov chains under a condition imposed to the maximal coefficient of correlation. The proofs are based on martingale techniques and a sharp lower bound estimate for the variance of partial sums. The results complement an important central limit theorem of Dobrushin based on the contraction coefficient.
We propose some backward-forward martingale decompositions for functions of reversible Markov chains. These decompositions are used to prove the functional CLT for reversible Markov chains with asymptotically linear variance of partial sums. We also provide a proof of the equivalence between asymptotic linearity of the variance and convergence of the integral of $1/(1-t)$ with respect to the associated spectral measure $ρ$. We also study the asymptotic behavior of linear processes having as innovations mean zero square integrable functions of stationary reversible Markov chains. We apply this …
Our aim is to find sufficient conditions for weak convergence of stochastic integrals with respect to the state occupation measure of a Markov chain. First, we study properties of the state indicator function and the state occupation measure of a Markov chain. In particular, we establish weak convergence of the state occupation measure under a scaling of the generator matrix. Then, relying on the connection between the state occupation measure and the Dynkin martingale related to the state indicator function, we provide sufficient conditions for weak convergence of stochastic integrals with re…
We consider a Markov chain that iteratively generates a sequence of random finite words in such a way that the $n^{\mathrm{th}}$ word is uniformly distributed over the set of words of length $2n$ in which $n$ letters are $a$ and $n$ letters are $b$: at each step an $a$ and a $b$ are shuffled in uniformly at random among the letters of the current word. We obtain a concrete characterization of the Doob-Martin boundary of this Markov chain. Writing $N(u)$ for the number of letters $a$ (equivalently, $b$) in the finite word $u$, we show that a sequence $(u_n)_{n \in \mathbb{N}}$ of finite words c…
We state and prove a quantitative version of the bounded difference inequality for geometrically ergodic Markov chains. Our proof uses the same martingale decomposition as \cite{MR3407208} but, compared to this paper, the exact coupling argument is modified to fill a gap between the strongly aperiodic case and the general aperiodic case.
We show fundamental properties of the Markov semigroup of recently proposed MCMC algorithms based on Piecewise-deterministic Markov processes (PDMPs) such as the Bouncy Particle Sampler, the Zig-Zag process or the Randomized Hamiltonian Monte Carlo method. Under assumptions typically satisfied in MCMC settings, we prove that PDMPs are Feller and that their generator admits the space of infinitely differentiable functions with compact support as a core. As we illustrate via martingale problems and a simplified proof of the invariance of target distributions, these results provide a fundamental …
