We study the exponential dissipation of entropic functionals for continuous time Markov chains and the associated convex Sobolev inequalities, including MLSI and Beckner inequalities. We propose a method that combines the Bakry Émery approach and coupling arguments, which we use as a probabilistic alternative to the discrete Bochner identities. The method is well suited to work in a non perturbative setting and we obtain new estimates for interacting random walks beyond the high temperature/weak interaction regime. In this framework, we also show that the exponential contraction of the Wasserstein distance along the semigroup implies MLSI. We also revisit classical examples often obtaining new inequalities and sometimes improving on the best known constants. In particular, we analyse the zero range dynamics, hardcore and Bernoulli-Laplace models and the Glauber dynamics for the Curie Weiss and Ising model.
This paper considers the Poisson equation for general state-space Markov chains in continuous time. The main purpose of this paper is to present specific bounds for the solutions of the Poisson equation for general state-space Markov chains. The solutions of the Poisson equation are unique in the sense that they are expressed in terms of a certain probabilistically interpretable solution (called the {\it standard solution}). Thus, we establish some specific bounds for the standard solution under the $f$-modulated drift condition (which is a kind of Foster-Lyapunov-type condition) and some mode…
This paper is devoted to probabilistic cellular automata (PCA) on $\mathbb{N}$, $\mathbb{Z}$ or $\mathbb{Z}/n\mathbb{Z}$, depending of two neighbors, with a general alphabet $E$ (finite or infinite, discrete or not). We study the following question: under which conditions does a PCA possess a Markov chain as invariant distribution? Previous results in the literature give some conditions on the transition matrix (for positive rate PCA) when the alphabet $E$ is finite. Here we obtain conditions on the transition kernel of PCA with a general alphabet $E$. In particular, we show that the existence…
The analytic properties of the Markov operator associated to a random walk are common tools in the study of the behaviour and some probabilistic features related to the walk. In this paper we consider a class of Markov operators which generalizes the class of compact Markov operators and we study some probabilistic properties of the associated random walk.
Let $T$ be the triangle in the plane with vertices $(0,0)$, $(0,1)$ and $(0,1)$. The convex hull of $(0,1)$, $(1,0)$ and $n$ independent random points uniformly distributed in $T$ is the random convex chain $T_n$. A three-term recursion for the probability generating function $G_n$ of the number $f_0(T_n)$ of vertices of $T_n$ is proved. Via the link to orthogonal polynomials it is shown that $G_n$ has precisely $n$ distinct real roots in $(-\infty,0]$ and that the sequence $p_k^{(n)}:=\mathbb{P}(f_0(T_n)=k)$, $k=1,\ldots,n$, is a Polya frequency (PF) sequence. A selection of probabilistic con…
We use here a particle system to prove a convergence result as well as a deviation inequality for solutions of granular media equation when the confinement potential and the interaction potential are no more uniformly convex. Proof is straightforward, simplifying deeply proofs of Carrillo-McCann-Villani \cite{CMV,CMV2} and completing results of Malrieu \cite{malrieu03} in the uniformly convex case. It relies on an uniform propagation of chaos property and a direct control in Wasserstein distance of solutions starting with different initial measures. The deviation inequality is obtained via a $…
In an influential paper, Courtois and Semal (1984) establish that when $G$ is an irreducible substochastic matrix for which $\sum_{n=0}^{\infty}G^n <\infty$, then the stationary distribution of any stochastic matrix $P\ge G$ can be expressed as a convex combination of the normalized rows of $(I-G)^{-1} = \sum_{n=0}^{\infty} G^n$. In this note, we give a short proof of this result that extends the theory to the countably infinite and continuous state space settings. This result plays an important role in obtaining error bounds in algorithms involving nearly decomposable Markov chains, and also …
In this paper, we study the existence and uniqueness of solutions for general fractional-time parabolic equations of mixture type, and their probabilistic representations in terms of the corresponding inverse subordinators with or without drifts. An explicit relation between occupation measure for Markov processes time-changed by inverse subordinator in open sets and that of the original Markov process in the open set is also given.
The Markov commutator associated to a finite Markov kernel P is the convex semigroup consisting of all Markov kernels commuting with P. Its interest comes from its relation with the hypergroup property and with the notion of Markovian duality by intertwining. In particular, it is shown that the discrete analogue of the Achour-Trim{è}che's theorem, asserting the preservation of non-negativity by the wave equations associated to certain Metropolis birth and death transition kernels, cannot be extended to all convex potentials. But it remains true for symmetric and monotone convex potentials. Key…
In this paper, we consider continuous-time Markov chains with a finite state space under nonlinear expectations. We define so-called Q-operators as an extension of Q-matrices or rate matrices to a nonlinear setup, where the nonlinearity is due to model uncertainty. The main result gives a full characterization of convex Q-operators in terms of a positive maximum principle, a dual representation by means of Q-matrices, continuous-time Markov chains under convex expectations and nonlinear ordinary differential equations. This extends a classical characterization of generators of Markov chains to…
Let $f$ be the density function associated to a matrix-exponential distribution of parameters $(α, T,s)$. By exponentially tilting $f$, we find a probabilistic interpretation which generalises the one associated to phase-type distributions. More specifically, we show that for any sufficiently large $λ\ge 0$, the function $x\mapsto \left(\int_0^\infty e^{-λr}f(r) dr\right)^{-1}e^{-λx}f(x)$ can be described in terms of a Markov jump process whose generator is tied to $T$. Finally, we show how to revert the exponential tilting in order to assign a probabilistic interpretation to $f$ itself.
We study the continuous-time evolution of the recombination equation of population genetics. This evolution is given by a differential equation that acts on a product probability space, and its solution can be described by a Markov chain on a set of partitions that converges to the finest partition. We study an explicit form of the law of this process by using a family of trees. We also describe the geometric decay rate to the finest partition and the quasi-stationary behavior of the Markov chain when conditioned on the event that the chain does not hit the limit.
In this paper we study homomorphisms of Probabilistic Regulatory Gene Networks(PRN) introduced in arXiv:math.DS/0603289 v1 13 Mar 2006. The model PRN is a natural generalization of the Probabilistic Boolean Networks (PBN), introduced by I. Shmulevich, E. Dougherty, and W. Zhang in 2001, that has been using to describe genetic networks and has therapeutic applications. In this paper, our main objectives are to apply the concept of homomorphism and $ε$-homomorphism of probabilistic regulatory networks to the dynamic of the networks. The meaning of $ε$ is that these homomorphic networks have simi…
We consider the problem of estimating the transition rate matrix of a continuous-time Markov chain from a finite-duration realisation of this process. We approach this problem in an imprecise probabilistic framework, using a set of prior distributions on the unknown transition rate matrix. The resulting estimator is a set of transition rate matrices that, for reasons of conjugacy, is easy to find. To determine the hyperparameters for our set of priors, we reconsider the problem in discrete time, where we can use the well-known Imprecise Dirichlet Model. In particular, we show how the limit of …
We present recent results on Piecewise Deterministic Markov Processes (PDMPs), involved in biological modeling. PDMPs, first introduced in the probabilistic literature by Davis (1984), are a very general class of Markov processes and are being increasingly popular in biological applications. They also give new interesting challenges from the theoretical point of view. We give here different examples on the long time behavior of switching Markov models applied to population dynamics, on uniform sampling in general branching models applied to structured population dynamic, on time scale separati…
Tries are among the most versatile and widely used data structures on words. They are pertinent to the (internal) structure of (stored) words and several splitting procedures used in diverse contexts ranging from document taxonomy to IP addresses lookup, from data compression (i.e., Lempel-Ziv'77 scheme) to dynamic hashing, from partial-match queries to speech recognition, from leader election algorithms to distributed hashing tables and graph compression. While the performance of tries under a realistic probabilistic model is of significant importance, its analysis, even for simplest memoryle…
We establish the equivalence of the analytic and probabilistic notions of subharmonicity in the framework of general symmetric Hunt processes on locally compact separable metric spaces, extending an earlier work of the first named author on the equivalence of the analytic and probabilistic notions of harmonicity. As a corollary, we prove a strong maximum principle for locally bounded finely continuous subharmonic functions in the space of functions locally in the domain of the Dirichlet form under some natural conditions.
The two-parameter Macdonald polynomials are a central object of algebraic combinatorics and representation theory. We give a Markov chain on partitions of k with eigenfunctions the coefficients of the Macdonald polynomials when expanded in the power sum polynomials. The Markov chain has stationary distribution a new two-parameter family of measures on partitions, the inverse of the Macdonald weight (rescaled). The uniform distribution on permutations and the Ewens sampling formula are special cases. The Markov chain is a version of the auxiliary variables algorithm of statistical physics. Prop…
In this work, we study discrete-time Markov decision processes (MDPs) under constraints with Borel state and action spaces and where all the performance functions have the same form of the expected total reward (ETR) criterion over the infinite time horizon. One of our objective is to propose a convex programming formulation for this type of MDPs. It will be shown that the values of the constrained control problem and the associated convex program coincide and that if there exists an optimal solution to the convex program then there exists a stationary randomized policy which is optimal for th…
We study a new notion of Ricci curvature that applies to Markov chains on discrete spaces. This notion relies on geodesic convexity of the entropy and is analogous to the one introduced by Lott, Sturm, and Villani for geodesic measure spaces. In order to apply to the discrete setting, the role of the Wasserstein metric is taken over by a different metric, having the property that continuous time Markov chains are gradient flows of the entropy. Using this notion of Ricci curvature we prove discrete analogues of fundamental results by Bakry--Emery and Otto--Villani. Furthermore we show that Ricc…
In recent years probabilistic model checking has become an important area of research because of the diffusion of computational systems of stochastic nature. Despite its great success, standard probabilistic model checking suffers the limitation of requiring a sharp specification of the probabilities governing the model behaviour. The theory of imprecise probabilities offers a natural approach to overcome such limitation by a sensitivity analysis with respect to the values of these parameters. However, only extensions based on discrete-time imprecise Markov chains have been considered so far f…
Using a mixture of classical and probabilistic techniques we investigate the convexity of solutions to the elliptic pde associated with a certain generalized Ornstein-Uhlenbeck process.
Consider a non-autonomous continuous-time linear system in which the time-dependent matrix determining the dynamics is piecewise constant and takes finitely many values $A_1, \dotsc, A_N$. This paper studies the equality cases between the maximal Lyapunov exponent associated with the set of matrices $\{A_1, \dotsc, A_N\}$, on the one hand, and the corresponding ones for piecewise deterministic Markov processes with modes $A_1, \dotsc, A_N$, on the other hand. A fundamental step in this study consists in establishing a result of independent interest, namely, that any sequence of Markov processe…
