We study the Classical Probability analogue of the dilations of a quantum dynamical semigroup in Quantum Probability. Given a (not necessarily homogeneous) Markov chain in discrete time in a finite state space E, we introduce a second system, an environment, and a deterministic invertible time-homogeneous global evolution of the system E with this environment such that the original Markov evolution of E can be realized by a proper choice of the initial random state of the environment. We also compare this dilations with the dilations of a quantum dynamical semigroup in Quantum Probability: given a classical Markov semigroup, we show that it can be extended to a quantum dynamical semigroup for which we can find a quantum dilation to a group of *-automorphisms admitting an invariant abelian subalgebra where this quantum dilation gives just our classical dilation.
Given a finite state space E, we build a universal dilation for all possible discrete time Markov chains on E, homogeneous or not: we introduce a second system (an ``environment'') and a deterministic invertible time-homogeneous global evolution of the system E with this environment such that any Markov evolution of E can be realized by a proper choice of the initial (random) state of the environment, which therefore determines the transition probabilities of the system. We also compare this dilation with the quantum dilations of a Quantum Dynamical Semigroup: given a Classical Markov Semigrou…
We study the Classical Probability analogue of the dilations of a quantum dynamical semigroup defined in Quantum Probability via quantum stochastic differential equations. Given a homogeneous Markov chain in continuous time in a finite state space E, we introduce a second system, an environment, and a deterministic invertible time-homogeneous global evolution of the system E with this environment such that the original Markov evolution of E can be realized by a proper choice of the initial random state of the environment. We also compare this dilations with the dilations of a quantum dynamical…
We present, for the first time, the result (from 2008) that (normal, strongly continuous) Markov semigroups on $\mathscr{B}(G)$ ($G$ a separable Hilbert space) admit a Hudson-Parthasarathy dilation (that is, a dilation to a cocycle perturbation of a noise) if and only if the Markov semigroup is spatial (that is, if it dominates an elementary CP-semigroup). The proof is by general abstract nonsense (taken from Arveson's classification of $E_0$-semigroups on $\mathscr{B}(H)$ by Arveson systems up to cocycle conjugacy) and not, as usual, by constructing the cocycle as a solution of a quantum stoc…
This note presents a simple proof of the monotonicity of the invariant distribution of a discrete Markov chain with a finite state space. This answers a question recently raised by David Siegmund.
In this note, we are concerned with the subgeometric rate of convergence of a Markov chain with discrete time parameter to its invariant measure in the $f$-norm. We clarify how three typical subgeometric rates of convergence are inherited under a discrete time version of Bochner's subordination. The crucial point is to establish the corresponding moment estimates for discrete time subordinators under some reasonable conditions on the underlying Bernstein function.
The asymptotic variance is an important criterion to evaluate the performance of Markov chains, especially for the central limit theorems. We give the variational formulas for the asymptotic variance of discrete-time (non-reversible) Markov chains on general state space. The variational formulas provide many applications, extending the classical Peskun's comparison theorem to non-reversible Markov chains, and obtaining several comparison theorems between Markov chains with various perturbations.
In the last years, many authors studied a class of continuous time semi-Markov processes obtained by time-changing Markov processes by hitting times of independent subordinators. Such processes are governed by integro-differential convolution equations of generalized fractional type. The aim of this paper is to develop the discrete-time version of such a theory. We show that a class of discrete-time semi-Markov chains can be seen as time-changed Markov chains and we obtain governing convolution type equations. Such processes converge weakly to those in continuous time under suitable scaling li…
In this paper we study the spectral properties of Markov-operator on $L^{2}$-spaces. Lawler and Sokal (Trans. Amer. Math. Soc., 1988, 309, pp. 557-580) used isoperimetric constants for discrete and continuous time Markov chains to obtain a spectral gap at 1. For time discrete Markov chains this does not exclude periodic behavior. We define a new constant measuring the distance from periodicity and give necessary and sufficient conditions for the existence of a global spectral gap in terms of this constant.
This paper studies the augmented truncation of discrete-time block-monotone Markov chains under geometric drift conditions. We first present a bound for the total variation distance between the stationary distributions of an original Markov chain and its augmented truncation. We also obtain such error bounds for more general cases where an original Markov chain itself may not be block-monotone but is block-wise dominated by a block-monotone Markov chain. Finally we discuss the application of our results to GI/G/1-type Markov chains.
In this paper, we study quasi-stationary distributions of nonlinearly perturbed semi-Markov processes in discrete time. This type of distributions is of interest for the analysis of stochastic systems which have finite lifetimes, but are expected to persist for a long time. We obtain asymptotic power series expansions for quasi-stationary distributions and it is shown how the coefficients in these expansions can be computed from a recursive algorithm. As an illustration of this algorithm, we present a numerical example for a discrete time Markov chain.
The class of nonlinear Markov processes is characterized by the dependence of the current state of the process on its current distribution in addition to the dependence on the previous state. Due to this feature, these processes are characterized by complex limit behavior and ergodic properties, for which the usual criteria for Markov processes are not sufficient. Being a subclass of nonlinear Markov processes, nonlinear Markov chains have inherited these features. In this paper, conditions for the fulfillment of the central limit theorem for homogeneous nonlinear Markov chains in discrete tim…
General characterizations of ergodic Markov chains have been developed in considerable detail. In this paper, we study the transience for discrete-time Markov chains on general state spaces, including the geometric transience and algebraic transience. Criteria are presented through establishing the drift condition and considering the first return time. As an application, we give explicit criteria for the random walk on the half line and the skip-free chain on nonnegative integers.
We show that representations of the Thompson group $F$ in the automorphisms of a noncommutative probability space yield a large class of bilateral stationary noncommutative Markov processes. As a partial converse, bilateral stationary Markov processes in tensor dilation form yield representations of $F$. As an application, and building on a result of Kümmerer, we canonically associate a representation of $F$ to a bilateral stationary Markov process in classical probability.
The paper studies the time-homogeneous two-state Markov chains; the states are assumed to be binary symbols 0 and 1. The higher-order absolute differences taken from progressive states of a given chain are considered. A discrete capacity of subsets of natural series is defined and a limiting theorem for these differences, formulated in terms of Wiener criterion type relation, is presented.
Recombination in discrete and continuous time from the viewpoint of Markov embedding
Technische Fakultät, Universität Bielefeld（比勒费尔德大学技术学院）；Fakultät für Mathematik, Universität Bielefeld（比勒费尔德大学数学学院）；School of Natural Sciences, Discipline of Mathematics, University of Tasmania（塔斯马尼亚大学自然科学学院数学系）
The article is devoted to the estimation of the rate of convergence of integral functionals of a Markov process. Under the assumption that the given Markov process admits a transition probability density which is differentiable in $t$ and the derivative has an integrable upper bound of a certain type, we derive the accuracy rates for strong and weak approximations of the functionals by Riemannian sums. Some examples are provided.
This paper studies the last-column-block-augmented northwest-corner truncation (LC-block-augmented truncation, for short) of discrete-time block-monotone Markov chains under subgeometric drift conditions. The main result of this paper is to present an upper bound for the total variation distance between the stationary probability vectors of a block-monotone Markov chain and its LC-block-augmented truncation. The main result is extended to Markov chains that themselves may not be block monotone but are block-wise dominated by block-monotone Markov chains satisfying modified drift conditions. Fi…
In this paper, we study mixed power-exponential moment functionals of nonlinearly perturbed semi-Markov processes in discrete time. Conditions under which the moment functionals of interest can be expanded in asymptotic power series with respect to the perturbation parameter are given. We show how the coefficients in these expansions can be computed from explicit recursive formulas. In particular, the results of the present paper have applications for studies of quasi-stationary distributions.
Discrete convex Sobolev inequalities and Beckner inequalities are derived for time-continuous Markov chains on finite state spaces. Beckner inequalities interpolate between the modified logarithmic Sobolev inequality and the Poincaré inequality. Their proof is based on the Bakry-Emery approach and on discrete Bochner-type inequalities established by Caputo, Dai Pra, and Posta and recently extended by Fathi and Maas for logarithmic entropies. The abstract result for convex entropies is applied to several Markov chains, including birth-death processes, zero-range processes, Bernoulli-Laplace mod…
In the paper, we study a new rate of convergence estimate for homogeneous discrete-time nonlinear Markov chains based on the Markov-Dobrushin condition. This result generalizes the convergence estimates for any positive number of transition steps. An example of a class such a process provided indicates that such types of estimates considering several transition steps may be applicable when one transition can not guarantee any convergence. Moreover, a better estimate can be obtained for a higher number of transitions steps. A law of large numbers is presented for a class of ergodic nonlinear Ma…
We consider a discrete time semi-Markov process where the characteristics defining the process depend on a small perturbation parameter. It is assumed that the state space consists of one finite communicating class of states and, in addition, one absorbing state. Our main object of interest is the asymptotic behaviour of the joint probabilities of the position of the semi-Markov process and the event of non-absorption as time tends to infinity and the perturbation parameter tends to zero. The main result gives exponential expansions of these probabilities together with an recursive algorithm f…
Let $\{X_n\}_{n\in\N}$ be a Markov chain on a measurable space $\X$ with transition kernel $P$ and let $V:\X\r[1,+\infty)$. The Markov kernel $P$ is here considered as a linear bounded operator on the weighted-supremum space $\cB_V$ associated with $V$. Then the combination of quasi-compactness arguments with precise analysis of eigen-elements of $P$ allows us to estimate the geometric rate of convergence $ρ_V(P)$ of $\{X_n\}_{n\in\N}$ to its invariant probability measure in operator norm on $\cB_V$. A general procedure to compute $ρ_V(P)$ for discrete Markov random walks with identically dist…
We work out some relations between duality and intertwining in the context of discrete Markov chains, fixing up the background of previous relations first established for birth and death chains and their Siegmund duals. In view of the results, the monotone properties resulting from the Siegmund dual of birth and death chains are revisited in some detail, with emphasis on the non neutral Moran model. We also introduce an ultrametric type dual extending the Siegmund kernel. Finally we discuss the sharp dual, following closely the Diaconis-Fill study.
In probability theory, equalities are much less than inequalities. In this paper, we find a series of equalities which characterize the symmetry of the forming times of a family of similar cycles for discrete-time and continuous-time Markov chains. Moreover, we use these cycle symmetries to study the circulation fluctuations for Markov chains. We prove that the empirical circulations of a family of cycles passing through a common state satisfy a large deviation principle with a rate function which has an highly non-obvious symmetry. Finally, we discuss the applications of our work in statistic…
We build optimal exponential bounds for the probabilities of large deviations of sums \sum_{k=1}^nf(X_k) where (X_k) is a finite reversible Markov chain and f is an arbitrary bounded function. These bounds depend only on the stationary mean E_πf, the end-points of the support of f, the sample size n and the second largest eigenvalue λof the transition matrix.
We study perturbation theory and uniform ergodicity for discrete-time Markov chains on general state spaces in terms of the uniform moments of the first hitting times on some set. The methods we adopt are different from previous ones. For reversible and non-negative definite Markov chains, we first investigate the geometrically ergodic convergence rates. Based on the estimates, together with a first passage formula, we then get the convergence rates in uniform ergodicity. If the transition kernel $P$ is only reversible, we transfer to study the two-skeleton chain with the transition kernel $P^…
We construct and study branching Markov processes on the space of finite configurations of the state space of a given standard process, controlled by a branching kernel and a killing one. In particular, we may start with a superprocess, obtaining a branching process with state space the finite configurations of positive finite measures on a topological space. A main tool in proving the path regularity of the branching process is the existence of convenient superharmonic functions having compact level sets, allowing the use of appropriate potential theoretical methods.
Let $X_n$ be a discrete time Markov chain with state space $S$ (countably infinite, in general) and initial probability distribution $μ^{(0)} = (P(X_0=i_1),P(X_0=i_2),\cdots,)$. What is the probability of choosing in random some $k \in \mathbb{N}$ with $k \leq n$ such that $X_k = j$ where $j \in S$? This probability is the average $\frac{1}{n} \sum_{k=1}^n μ^{(k)}_j$ where $μ^{(k)}_j = P(X_k = j)$. In this note we will study the limit of this average without assuming that the chain is irreducible, using elementary mathematical tools. Finally, we study the limit of the average $\frac{1}{n} \sum…
The main topic of these notes are Markov loops, studied in the context of continuous time Markov chains on discrete state spaces. We refer to [1] and [2] for the short "history" of the subject. In contrast with these references, symmetry is not assumed, and more attention is given to the infinite case. All results are presented in terms of the semigroup generator. In comparison with [1], some delicate proofs are given in more details or with a better method. We focus mostly on properties of the (multi)occupation field but also included some results about loop clusters (see [3] in the symmetric…
We consider a random walk on a discrete connected graph having some infinite branches plus finitely many vertices with finite degrees. We find the generator of a strong stationary dual in the sense of Fill, and use it to find some equivalent condition to the existence of a strong stationary time. This strong stationary dual process lies in the set of connected compact sets of the compactification of the graph. When this graph is Z, this is simply the set of (possibly infinite) segments of Z.
We are interested in quasi-stationarity and quasi-ergodicity when the absorbing boundary is moving. First we show that, in the moving boundary case, the quasi-stationary distribution and the quasi-limiting distribution are not well-defined when the boundary is oscillating periodically. Then we show the existence of a quasi-ergodic distribution for any discrete-time irreducible Markov chain defined on a finite space state in the fixed boundary case. Finally we use this last result to show the quasi-ergodicity in the moving boundary case.
In this paper, we consider general Markov chains (MC), specified by the transition probability (kernel) $ P (x, E) $, finitely additive in the second argument. Such MC are studied within the framework of the functional operator treatment. The state space (phase space) of the MC $ X $ has any cardinality, and the sigma-algebra $ Σ$ is discrete, i.e. is the set of all subsets in $ X $. This construction of the phase space $ (X, Σ) $ allows us to decompose the Markov kernel $ P (x, E) $ into the sum of two components - countably additive and purely finitely additive in the second argument and mea…
We prove sharp estimates on heat kernels and Green functions for subordinate Markov processes with both discrete an continuous time, under relatively weak assumptions about original processes as well as Laplace exponents of subordinators. We also show stability of heat kernel estimates in the discrete settings.
Our purpose is to explore, in the context of loop ensembles on finite graphs, the relations between combinatorial group theory, loops topology, loop measures, and signatures of discrete paths. We determine the distributions of the loop homotopy class, and of the first and second homologies, defined by the lower central series of the fundamental group.
In the present paper we investigate the $L_1$-weak ergodicity of nonhomogeneous discrete Markov processes with general state spaces. Note that the $L_1$-weak ergodicity is weaker than well-known weak ergodicity. We provide a necessary and sufficient condition for such processes to satisfy the $L_1$-weak ergodicity. Moreover, we apply the obtained results to establish $L_1$-weak ergodicity of discrete time quadratic stochastic processes. As an application of the main result, certain concrete examples are also provided.
We consider mean field games with ergodic cost in the framework of a general discrete time controlled Markov processes. The state space of the processes is given by a general $σ$-compact Polish space. Under certain conditions, we show the existence of a mean field game equilibrium. We also study the $N$-person game where the players interacts with each other via their empirical measure. We show that the $N$-person game has Nash equilibrium and as $N$ tends to infinity the equilibria converge to a mean field game solution.
In this paper, we present a discrete-type approximation scheme to solve continuous-time optimal stopping problems based on fully non-Markovian continuous processes adapted to the Brownian motion filtration. The approximations satisfy suitable variational inequalities which allow us to construct $ε$-optimal stopping times and optimal values in full generality. Explicit rates of convergence are presented for optimal values based on reward functionals of path-dependent SDEs driven by fractional Brownian motion. In particular, the methodology allows us to design concrete Monte-Carlo schemes for no…
A semi-Markov process is one that changes states in accordance with a Markov chain but takes a random amount of time between changes. We consider the generalisation to semi-Markov processes of the classical Lamperti law for the occupation time of a two-state Markov process. We provide an explicit expression in Laplace space for the distribution of an arbitrary linear combination of the occupation times in the various states of the process. We discuss several consequences of this result. In particular, we infer the limiting distribution of this quantity rescaled by time in the long-time scaling…
We show that if the random walk on a graph has positive coarse Ricci curvature in the sense of Ollivier, then the stationary measure satisfies a W^1 transport-entropy inequality. Peres and Tetali have conjectured a stronger consequence, that a modified log-Sobolev inequality (MLSI) should hold, in analogy with the setting of Markov diffusions. We discuss how our entropy interpolation approach suggests a natural attack on the MLSI conjecture.
This work establishes sufficient conditions for existence of saddle points in discrete Markov games. The result reveals the relation between dynamic games and static games using dynamic programming equations. This result enables us to prove existence of saddle points of non-separable stochastic differential games of regime-switching diffusions under appropriate conditions.
We consider the problem of filtering an unseen Markov chain from noisy observations, in the presence of uncertainty regarding the parameters of the processes involved. Using the theory of nonlinear expectations, we describe the uncertainty in terms of a penalty function, which can be propagated forward in time in the place of the filter.
This paper examines a discrete-time queuing system with applications to telecommunications traffic. The arrival process is a particular Markov modulated process which belongs to the class of discrete batched Markovian arrival processes. The server process is a single server deterministic queue. A closed form exact solution is given for the expected queue length and delay. A simple system of equations is given for the probability of the queue exceeding a given length.
The first aim of this paper is to introduce a class of Markov chains on $\mathbb{Z}_+$ which are discrete self-similar in the sense that their semigroups satisfy an invariance property expressed in terms of a discrete random dilation operator. After showing that this latter property requires the chains to be upward skip-free, we first establish a gateway relation, a concept introduced in [26], between the semigroup of such chains and the one of spectrally negative self-similar Markov processes on $\mathbb{R}_+$. As a by-product, we prove that each of these Markov chains, after an appropriate s…
We investigate some particular completely positive maps which admit a stable commutative Von Neumann subalgebra. The restriction of such maps to the stable algebra is then a Markov operator. In the first part of this article, we propose a recipe in order to find a quantum extension of a given Markov operator in the above sense. We show that the existence of such an extension is linked with the existence of a special form of dilation for the Markov operator studied by Attal in \cite{Att1}, reducing the problem to the extension of dynamical system. We then apply our method to the same problem in…
By considering special sampling of discrete scale invariant (DSI) processes we provide a sequence which is in correspondence to multi-dimensional self-similar process. By imposing Markov property we show that the covariance functions of such discrete scale invariant Markov (DSIM) sequences are characterized by variance, and covariance of adjacent samples in the first scale interval. We also provide a theoretical method for estimating spectral density matrix of corresponding multi-dimensional self-similar Markov process. Some examples such as simple Brownian motion with drift and scale invarian…
Evolution algebras are non-associative algebras. In this work we provide an extension of this class of algebras, in the context of Hilbert spaces, capable to deal with infinite-dimensional spaces. We illustrate the applicability of our approach by discussing a connection with discrete-time Markov chains with countable state space.
