Characterizing whether a Markov process of discrete random variables has an homogeneous continuous-time realization is a hard problem. In practice, this problem reduces to deciding when a given Markov matrix can be written as the exponential of some rate matrix (a Markov generator). This is an old question known in the literature as the embedding problem (Elfving37), which has been only solved for matrices of size $2\times 2$ or $3\times 3$. In this paper, we address this problem and related questions and obtain results in two different lines. First, for matrices of any size, we give a bound on the number of Markov generators in terms of the spectrum of the Markov matrix. Based on this, we establish a criterion for deciding whether a generic Markov matrix (different eigenvalues) is embeddable and propose an algorithm that lists all its Markov generators. Then, motivated and inspired by recent results on substitution models of DNA, we focus in the $4\times 4$ case and completely solve the embedding problem for any Markov matrix. The solution in this case is more concise as the embeddability is given in terms of a single condition.
A Markov matrix is embeddable if it can represent a homogeneous continuous-time Markov process. It is well known that if a Markov matrix has real and pairwise-different eigenvalues, then the embeddability can be determined by checking whether its principal logarithm is a rate matrix or not. The same holds for Markov matrices close enough to the identity matrix or that rule a Markov process subjected to certain restrictions. In this paper we prove that this criterion cannot be generalized and we provide open sets of Markov matrices that are embeddable and whose principal logarithm is a not a ra…
Consider an nxn random matrix X with i.i.d. nonnegative entries with bounded density, mean m, and finite positive variance sigma^2. Let M be the nxn random Markov matrix with i.i.d. rows obtained from X by dividing each row of X by its sum. In particular, when X11 follows an exponential law, then M belongs to the Dirichlet Markov Ensemble of random stochastic matrices. Our main result states that with probability one, the counting probability measure of the complex spectrum of n^(1/2)M converges weakly as n tends to infinity to the uniform law on the centered disk of radius sigma/m. The bounde…
The practically important classes of equal-input and of monotone Markov matrices are revisited, with special focus on embeddability, infinite divisibility, and mutual relations. Several uniqueness results for the classic Markov embedding problem are obtained in the process. To achieve our results, we need to employ various algebraic and geometric tools, including commutativity, permutation invariance and convexity. Of particular relevance in several demarcation results are Markov matrices that are idempotents.
We investigate the spectral distribution of large sample covariance matrices with independent columns and entries in the columns that stem from Markov chains. We characterize the limiting spectral densities by their moments. Correspondingly, the proof is based on a moment method.
We give an account of some results, both old and new, about any $n\times n$ Markov matrix that is embeddable in a one-parameter Markov semigroup. These include the fact that its eigenvalues must lie in a certain region in the unit ball. We prove that a well-known procedure for approximating a non-embeddable Markov matrix by an embeddable one is optimal in a certain sense.
Let $(ξ_j)_{j\ge1} $, be a non-stationary Markov chain with phase space $X$ and let $\mathfrak{g}_j:\,X\mapsto\mathrm{SL}(m,\mathbb{R})$ be a sequence of functions on $X$ with values in the unimodular group. Set $g_j=\mathfrak{g}_j(ξ_j)$ and denote by $S_n=g_n\ldots g_1$, the product of the matrices $g_j$. We provide sufficient conditions for exponential growth of the norm $\|S_n\|$ when the Markov chain is not supposed to be stationary. This generalizes the classical theorem of Furstenberg on the exponential growth of products of independent identically distributed matrices as well as its ext…
This paper explicitly details the relation between $M$-matrices, nonnegative roots of nonnegative matrices, and the embedding problem for finite-state stationary Markov chains. The set of nonsingular nonnegative matrices with arbitrary nonnegative roots is shown to be the closure of the set of matrices with matrix roots in $\mathcal{IM}$. The methods presented here employ nothing beyond basic matrix analysis, however it answers a question regarding $M$-matrices posed over 30 years ago and as an application, a new characterization of the set of all embeddable stochastic matrices is obtained as …
In this note, we characterize the embeddability of generic Kimura 3ST Markov matrices in terms of their eigenvalues. As a consequence, we are able to compute the volume of such matrices relative to the volume of all Markov matrices within the model. We also provide examples showing that, in general, mutation rates are not identifiable from substitution probabilities. These examples also illustrate that symmetries between mutation probabilities do not necessarily arise from symmetries between the corresponding mutation rates.
We present an alternative proof of asymptotic freeness of independent sample covariance matrices, when the dimension and the sample size grow at the same rate, by embedding these matrices into Wigner matrices of a larger order and using asymptotic freeness of independent Wigner and deterministic matrices.
The embedding problem for Markov chains is a famous problem in probability theory and only partial results are available up till now. In this paper, we propose a variant of the embedding problem called the reversible embedding problem which has a deep physical and biochemical background and provide a complete solution to this new problem. We prove that the reversible embedding of a stochastic matrix, if it exists, must be unique. Moreover, we obtain the sufficient and necessary conditions for the existence of the reversible embedding and provide an effective method to compute the reversible em…
We equip the polytope of $n\times n$ Markov matrices with the normalized trace of the Lebesgue measure of $\mathbb{R}^{n^2}$. This probability space provides random Markov matrices, with i.i.d. rows following the Dirichlet distribution of mean $(1/n,...,1/n)$. We show that if $\bM$ is such a random matrix, then the empirical distribution built from the singular values of$\sqrt{n} \bM$ tends as $n\to\infty$ to a Wigner quarter--circle distribution. Some computer simulations reveal striking asymptotic spectral properties of such random matrices, still waiting for a rigorous mathematical analysis…
In order to approximate a continuous time stochastic process by discrete time Markov chains one has several options to embed the Markov chains into continuous time processes. On the one hand there is the Markov embedding, which uses exponential waiting times. On the other hand each Skorokhod topology naturally suggests a certain embedding. These are the step function embedding for $J_1$, the linear interpolation embedding for $M_1$, the multi step embedding for $J_2$ and a more general embedding for $M_2$. We show that the convergence of the step function embedding in $J_1$ implies the converg…
In this paper, we present reversibility preserving operations on Markov chain transition matrices. Simple row and column operations allow us to create new reversible transition matrices and yield an easy method for checking a Markov chain for reversibility.
The representation problem of finite-dimensional Markov matrices in Markov semigroups is revisited, with emphasis on concrete criteria for matrix subclasses of theoretical or practical relevance, such as equal-input, circulant, symmetric or doubly stochastic matrices. Here, we pay special attention to various algebraic properties of the embedding problem, and discuss the connection with the centraliser of a Markov matrix.
Given a matrix of distribution functions and a quasi-stochastic matrix, i.e. an irreducible nonnegative matrix with maximal eigenvalue one and associated unique positive left and right eigenvectors, the article studies the properties of an associated matrix renewal measure and a related integral equation. Unlike earlier work this is done by a purely probabilistic approach based on a simple harmonic transform. Main results include Markov renewal-type theorems and a Stone-type decomposition under an absolute continuity condition. Three applications are given at the end of the paper.
Recombination in discrete and continuous time from the viewpoint of Markov embedding
Technische Fakultät, Universität Bielefeld（比勒费尔德大学技术学院）；Fakultät für Mathematik, Universität Bielefeld（比勒费尔德大学数学学院）；School of Natural Sciences, Discipline of Mathematics, University of Tasmania（塔斯马尼亚大学自然科学学院数学系）
Random sampling of large Markov matrices with a tunable spectral gap, a nonuniform stationary distribution, and a nondegenerate limiting empirical spectral distribution (ESD) is useful. Fix $c>0$ and $p>0$. Let $A_n$ be the adjacency matrix of a random graph following $\mathrm{G}(n, p/n)$, known as the Erdős-Rényi distribution. Add $c/n$ to each entry of $A_n$ and then normalize its rows. It is shown that the resulting Markov matrix has the desired properties. Its ESD weakly converges in probability to a symmetric nondegenerate distribution, and its extremal eigenvalues, other than 1, fall in …
We consider Markov processes of cubic stochastic (in a fixed sense) matrices which are also called quadratic stochastic process (QSPs). A QSP is a particular case of a continuous-time dynamical system whose states are stochastic cubic matrices satisfying an analogue of the Kolmogorov-Chapman equation (KCE). Since there are several kinds of multiplications between cubic matrices we have to fix first a multiplication and then consider the KCE with respect to the fixed multiplication. Moreover, the notion of stochastic cubic matrix also varies depending on the real models of application. The exis…
Let S be a denumerable state space and let P be a transition probability matrix on S. If a denumerable set M of nonnegative matrices is such that the sum of the matrices is equal to P, then we call M a partition of P. Let K denote the set of probability vectors on S. To every partition M of P we can associate a transition probability function on K defined in such a way that if p in K and m in M are such that ||pm|| > 0, then, with probability ||pm|| the vector p is transferred to the vector pm/||pm||. Here ||.|| denotes the l_1-norm. In this paper we investigate convergence in distribution for…
We develop an explicit non-randomized solution to the Skorokhod embedding problem in an abstract setup of signed functionals of Markovian excursions. Our setting allows to solve the Skorokhod embedding problem, in particular, for diffusions and their (signed, scaled) age processes, for Azema's martingale, for spectrally one-sided Levy processes and their reflected versions, for Bessel processes of dimension smaller than 2, and for their age processes, as well as for the age process of excursions of Cox-Ingersoll-Ross processes. This work is a continuation and an important generalization of Obl…
The aim of this work is to provide a rigorous mathematical analysis of a stochastic concatenation model presented by Sobottka and Hart (2011) which allows approximation of the first-order stochastic structure in bacterial DNA by means of a stationary Markov chain. Two probabilistic constructions that rigorously formalize the model are presented. Necessary and sufficient conditions for a Markov chain to be generated by the model are given, as well as the theoretical background needed for designing new algorithms for statistical analyses of real bacterial genomes. It is shown that the model enco…
We present a coarse-graining (or model order reduction) procedure for stochastic matrices by clustering. The method is consistent with the natural structure of Markov theory, preserving positivity and mass, and does not rely on any tools from Hilbert space theory. The reconstruction is provided by a generalized Penrose-Moore inverse of the coarse-graining operator incorporating the inhomogeneous invariant measure of the Markov matrix. As we show, the method provides coarse-graining and reconstruction also on the level of tensor spaces, which is consistent with the notion of an incidence matrix…
We prove a concentration phenomenon on the empirical eigenvalue distribution (EED) of the principal submatrix in a random hermitian matrix whose distribution is invariant under unitary conjugacy; for example, this class includes GUE (Gaussian Unitary Ensemble) and Wishart matrices. More precisely, if the EED of the whole matrix converges to some deterministic probability measure $\mathfrak{m}$, then its fluctuation from the EED of the principal submatrix, after a rescaling, concentrates at the Rayleigh measure (in general, a Schwartz distribution) associated with $\mathfrak{m}$ by the Markov--…
We provide a characterisation of the continuous-time Markov models where the Markov matrices from the model can be parameterised directly in terms of the associated rate matrices (generators). That is, each Markov matrix can be expressed as the sum of the identity matrix and a rate matrix from the model. We show that the existence of an underlying Jordan algebra provides a sufficient condition, which becomes necessary for (so-called) linear models. We connect this property to the well-known uniformization procedure for continuous-time Markov chains by demonstrating that the property is equival…
Let $(X_n \colon n\in\Z)$ be a two-sided recurrent Markov chain with fixed initial state $X_0$ and let $ν$ be a probability measure on its state space. We give a necessary and sufficient criterion for the existence of a non-randomized time $T$ such that $(X_{T+n} \colon n\in\Z)$ has the law of the same Markov chain with initial distribution $ν$. In the case when our criterion is satisfied we give an explicit solution, which is also a stopping time, and study its moment properties. We show that this solution minimizes the expectation of $ψ(T)$ in the class of all non-negative solutions, simulta…
We analyse the $\ell^2(π)$-convergence rate of irreducible and aperiodic Markov chains with $N$-band transition probability matrix $P$ and with invariant distribution $π$. This analysis is heavily based on: first the study of the essential spectral radius $r\_{ess}(P\_{|\ell^2(π)})$ of $P\_{|\ell^2(π)}$ derived from Hennion's quasi-compactness criteria; second the connection between the Spectral Gap property (SG$\_2$) of $P$ on $\ell^2(π)$ and the $V$-geometric ergodicity of $P$. Specifically, (SG$\_2$) is shown to hold under the condition $ α\_0 := \sum\_{{m}=-N}^N \limsup\_{i\rightarrow +\in…
In this paper, we consider continuous-time Markov chains with a finite state space under nonlinear expectations. We define so-called Q-operators as an extension of Q-matrices or rate matrices to a nonlinear setup, where the nonlinearity is due to model uncertainty. The main result gives a full characterization of convex Q-operators in terms of a positive maximum principle, a dual representation by means of Q-matrices, continuous-time Markov chains under convex expectations and nonlinear ordinary differential equations. This extends a classical characterization of generators of Markov chains to…
School of Computer Science, University of Birmingham（布里斯托尔大学计算机科学学院）；School of Electronics, Electrical Engineering, and Computer Science, Queen’s University Belfast（贝尔法斯特女王大学电子工程与计算机科学学院）
We study the convergence rate to stationarity for a class of exchangeable partition-valued Markov chains called cut-and-paste chains. The law governing the transitions of a cut-and-paste chain are determined by products of i.i.d. stochastic matrices, which describe the chain induced on the simplex by taking asymptotic frequencies. Using this representation, we establish upper bounds for the mixing times of ergodic cut-and-paste chains, and under certain conditions on the distribution of the governing random matrices we show that the "cutoff phenomenon" holds.
We study the limiting spectral measure of large symmetric random matrices of linear algebraic structure. For Hankel and Toeplitz matrices generated by i.i.d. random variables $\{X_k\}$ of unit variance, and for symmetric Markov matrices generated by i.i.d. random variables $\{X_{ij}\}_{j>i}$ of zero mean and unit variance, scaling the eigenvalues by $\sqrt{n}$ we prove the almost sure, weak convergence of the spectral measures to universal, nonrandom, symmetric distributions $γ_H$, $γ_M$ and $γ_T$ of unbounded support. The moments of $γ_H$ and $γ_T$ are the sum of volumes of solids related to …
For two metric spaces X and Y, say that X {threshold-embeds} into Y if there exist a number K > 0 and a family of Lipschitz maps $f_τ : X \to Y : τ> 0 \}$ such that for every $x,y \in X$, \[ d_X(x,y) \geq τ=> d_Y(f_τ(x),f_τ(y)) \geq \|φ_τ\|_{\Lip} τ/K \] where $\|f_τ\|_{\Lip}$ denotes the Lipschitz constant of $f_τ$. We show that if a metric space X threshold-embeds into a Hilbert space, then X has Markov type 2. As a consequence, planar graph metrics and doubling metrics have Markov type 2, answering questions of Naor, Peres, Schramm, and Sheffield. More generally, if a metric space X thresho…
We study model embeddability, which is a variation of the famous embedding problem in probability theory, when apart from the requirement that the Markov matrix is the matrix exponential of a rate matrix, we additionally ask that the rate matrix follows the model structure. We provide a characterisation of model embeddable Markov matrices corresponding to symmetric group-based phylogenetic models. In particular, we provide necessary and sufficient conditions in terms of the eigenvalues of symmetric group-based matrices. To showcase our main result on model embeddability, we provide an applicat…
We analyse the $\ell^2(π)$-convergence rate of irreducible and aperiodic Markov chains with $N$-band transition probability matrix $P$ and with invariant distribution $π$. This analysis is heavily based on: first the study of the essential spectral radius $r\_{ess}(P\_{|\ell^2(π)})$ of $P\_{|\ell^2(π)}$ derived from Hennion's quasi-compactness criteria; second the connection between the spectral gap property (SG$\_2$) of $P$ on $\ell^2(π)$ and the $V$-geometric ergodicity of $P$. Specifically, (SG$\_2$) is shown to hold under the condition \[α\_0 := \sum\_{{m}=-N}^N \limsup\_{i\rightarrow +\in…
Matryoshka dolls, the traditional Russian nesting figurines, are known world-wide for each doll's encapsulation of a sequence of smaller dolls. In this paper, we identify a large class of Markov process whose moments are easy to compute by exploiting the structure of a new sequence of nested matrices we call Matryoshkhan matrices. We characterize the salient properties of Matryoshkhan matrices that allow us to compute these moments in closed form at a specific time without computing the entire path of the process. This speeds up the computation of the Markov process moments significantly in co…
This paper quantifies the asymptotic order of the largest singular value of a centered random matrix built from the path of a Block Markov Chain (BMC). In a BMC there are $n$ labeled states, each state is associated to one of $K$ clusters, and the probability of a jump depends only on the clusters of the origin and destination. Given a path $X_0, X_1, \ldots, X_{T_n}$ started from equilibrium, we construct a random matrix $\hat{N}$ that records the number of transitions between each pair of states. We prove that if $ω(n) = T_n = o(n^2)$, then $\| \hat{N} - \mathbb{E}[\hat{N}] \| = Ω_{\mathbb{P…
Imprecise Transition Matrices for Markov Cohort Models: Lower and Upper Expectations with a Practical Health Economic Application
Medtronic Trading Sàrl（美敦力贸易公司）；Center for Evidence Synthesis in Health, Department of Health Services, Policy, and Practice, Brown University（布朗大学健康服务、政策与实践系健康证据综合中心）
Motivated by the theory of inhomogeneous Markov chains, we determine a sufficient condition for the convergence to 0 of a general product formed from a sequence of real or complex matrices. When the matrices have a common invariant subspace $H$, we give a sufficient condition for the convergence to 0 on $H$ of a general product. Our result is applied to obtain a condition for the weak ergodicity of an inhomogeneous Markov chain. We compare various types of contractions which may be defined for a single matrix, such as paracontraction, $l$--contraction, and $H$--contraction, where $H$ is an inv…
We prove a Chernoff-type bound for sums of matrix-valued random variables sampled via a regular (aperiodic and irreducible) finite Markov chain. Specially, consider a random walk on a regular Markov chain and a Hermitian matrix-valued function on its state space. Our result gives exponentially decreasing bounds on the tail distributions of the extreme eigenvalues of the sample mean matrix. Our proof is based on the matrix expander (regular undirected graph) Chernoff bound [Garg et al. STOC '18] and scalar Chernoff-Hoeffding bounds for Markov chains [Chung et al. STACS '12]. Our matrix Chernoff…
A spectral Favard theorem for bounded banded lower Hessenberg matrices that admit a positive bidiagonal factorization is found. The large knowledge on the spectral and factorization properties of oscillatory matrices leads to this spectral Favard theorem in terms of sequences of multiple orthogonal polynomials of types I and II with respect to a set of positive Lebesgue-Stieltjes~measures. Also a multiple Gauss quadrature is proven and corresponding degrees of precision are found. This spectral Favard theorem is applied to Markov chains with $(p+2)$-diagonal transition matrices, i.e. beyond bi…
