In this review-type paper written at the occasion of the Oberwolfach workshop {\em One-sided vs. Two-sided stochastic processes} (february 22-29, 2020), we discuss and compare Markov properties and generalisations thereof in more directions, as well as weaker forms of conditional dependence, again either in one or more directions. In particular, we discuss in both contexts various extensions of Markov chains and Markov fields and their properties, such as $g$-measures, Variable Length Markov Chains, Variable Neighbourhood Markov Fields, Variable Neighbourhood (Parsimonious) Random Fields, and Generalized Gibbs Measures.
We prove existence of weak martingale solutions satisfying an almost sure version of the energy inequality and which constitute a (almost sure) Markov process.
We consider almost upper semi-continuous processes defined on a finite Markov chain. The distributions of the functionals associated with the exit from a finite interval are studied. We also consider some modification of these processes.
We prove an invariance principle (functional central limit theorem) for a vector-valued additive functional of a Markov chain for almost every starting point with respect to an ergodic equilibrium distribution. The hypothesis is a moment bound on the resolvent.
We consider the almost semi-continuous processes defined on a finite Markov chain. The representation of the moment generating functions for the absolute maximum after achievement positive level and for the recovery time are obtained. Modified processes with two-step rate of negative jumps are investigated.
We consider a large class of piecewise expanding maps T of [0,1] with a neutral fixed point, and their associated Markov chain Y_i whose transition kernel is the Perron-Frobenius operator of T with respect to the absolutely continuous invariant probability measure. We give a large class of unbounded functions f for which the partial sums of f\circ T^i satisfy both a central limit theorem and a bounded law of the iterated logarithm. For the same class, we prove that the partial sums of f(Y_i) satisfy a strong invariance principle. When the class is larger, so that the partial sums of f\circ T^i…
In this article almost semi-continuous processes with stationary independent increments on a finite irreducible Markov chain are considered. For these processes the components of matrix factorization identity are concretely defined. On the basis of this concrete definition the relations for the distributions of extrema and distributions of their complements for the almost upper semi-continuous processes are established.
This work is devoted to the almost sure stabilization of adaptive control systems that involve an unknown Markov chain. The control system displays continuous dynamics represented by differential equations and discrete events given by a hidden Markov chain. Different from previous work on stabilization of adaptive controlled systems with a hidden Markov chain, where average criteria were considered, this work focuses on the almost sure stabilization or sample path stabilization of the underlying processes. Under simple conditions, it is shown that as long as the feedback controls have linear g…
In this paper we study the almost sure central limit theorem started from a point for additive functionals of a stationary and ergodic Markov chain via a martingale approximation in the almost sure sense. As a consequence we derive the quenched CLT for additive functionals or reversible Markov chains.
A transition matrix $[U_{i,j}]_{i,j\geq 0}$ on $\mathbb{N}$ is said to be almost upper triangular if $U_{i,j}\geq 0\Rightarrow j\geq i-1$, so that the increments of the corresponding Markov chains are at least $-1$; a transition matrix $[L_{i,j}]_{i,j\geq 0}$ is said to be almost lower triangular if $L_{i,j}\geq 0\Rightarrow j\leq i+1$, and then, the increments of the corresponding Markov chains are at most $+1$. In the present paper, we characterize the recurrence, positive recurrence and invariant distribution for the class of almost triangular transition matrices. The upper case appears to …
We study the global and local regularity properties of random wavelet series whose coefficients exhibit correlations given by a tree-indexed Markov chain. We determine the law of the spectrum of singularities of these series, thereby performing their multifractal analysis. We also show that almost every sample path displays an oscillating singularity at almost every point and that the points at which a sample path has at most a given Holder exponent form a set with large intersection.
Inspired by \citet{Berkes14} and \citet{Wu07}, we prove an almost sure invariance principle for stationary $β-$mixing stochastic processes defined on Hilbert space. Our result can be applied to Markov chain satisfying Meyn-Tweedie type Lyapunov condition and thus generalises the contraction condition in \citet[Example 2.2]{Berkes14}. We prove our main theorem by the big and small blocks technique and an embedding result in \citet{gotze2011estimates}. Our result is further applied to the ergodic Markov chain and functional autoregressive processes.
A general criterion is given for when a Markov chain trapped with probability p(x) in state x will be almost surely trapped. The quenched (state x is a trap forever with probability p(x)) and annealed (state x traps with probability p(x) on each visit) problems are shown to be equivalent.
The present paper has two goals. First to present a natural example of a new class of random fields which are the variable neighborhood random fields. The example we consider is a partially observed nearest neighbor binary Markov random field. The second goal is to establish sufficient conditions ensuring that the variable neighborhoods are almost surely finite. We discuss the relationship between the almost sure finiteness of the interaction neighborhoods and the presence/absence of phase transition of the underlying Markov random field. In the case where the underlying random field has no ph…
In this paper we give sufficient conditions for the almost sure central limit theorem started at a point, known under the name of quenched central limit theorem. This is achieved by using a new idea of conditioning with respect to both the past and the future of the Markov chain. As applications we provide a new sufficient projective conditions for the quenched CLT.
We investigate absorption, i.e., almost sure convergence to an absorbing state, in time-varying (non-homogeneous) discrete-time Markov chains with finite state space. We consider systems that can switch among a finite set of transition matrices, which we call the modes. Our analysis is focused on two properties: 1) almost sure convergence to an absorbing state under any switching, and 2) almost sure convergence to a desired set of absorbing states via a proper switching policy. We derive necessary and sufficient conditions based on the structures of the transition graphs of modes. More specifi…
We analyze the Markov property of solutions to the compressible Navier--Stokes system perturbed by a general multiplicative stochastic forcing. We show the existence of an almost sure Markov selection to the associated martingale problem. Our proof is based on the abstract framework introduced in [F. Flandoli, M. Romito: Markov selections for the 3D stochastic Navier--Stokes equations. Probab. Theory Relat. Fields 140, 407--458. (2008)]. A major difficulty arises from the fact, different from the incompressible case, that the velocity field is not continuous in time. In addition, it cannot be …
We study the limiting spectral measure of large symmetric random matrices of linear algebraic structure. For Hankel and Toeplitz matrices generated by i.i.d. random variables $\{X_k\}$ of unit variance, and for symmetric Markov matrices generated by i.i.d. random variables $\{X_{ij}\}_{j>i}$ of zero mean and unit variance, scaling the eigenvalues by $\sqrt{n}$ we prove the almost sure, weak convergence of the spectral measures to universal, nonrandom, symmetric distributions $γ_H$, $γ_M$ and $γ_T$ of unbounded support. The moments of $γ_H$ and $γ_T$ are the sum of volumes of solids related to …
In this paper, we discuss long-time behavior of sample paths for a wide range of regime-switching diffusions. Firstly, almost sure asymptotic stability is concerned (i) for regime-switching diffusions with finite state spaces by the Perron-Frobenius theorem, and, with regard to the case of reversible Markov chain, via the principal eigenvalue approach; (ii) for regime-switching diffusions with countable state spaces by means of a finite partition trick and an M-Matrix theory. We then apply our theory to study the stabilization for linear switching models. Several examples are given to demonstr…
For a polynomial $P_n$ of degree $n$, Bernstein's inequality states that $\|P_n'\| \le n \|P_n\|$ for all $L^p$ norms on the unit circle, $0<p\le\infty,$ with equality for $P_n(z)= c z^n.$ We study this inequality for random polynomials, and show that the expected (average) and almost sure value of $\Vert P_n' \Vert/\Vert P_n\Vert$ is often different from the classical deterministic upper bound $n$. In particular, for circles of radii less than one, the ratio $\Vert P_n' \Vert/\Vert P_n\Vert$ is almost surely bounded as $n$ tends to infinity, and its expected value is uniformly bounded for all…
We consider a class of discrete time Markov chains with state space [0,1] and the following dynamics. At each time step, first the direction of the next transition is chosen at random with probability depending on the current location. Then the length of the jump is chosen independently as a random proportion of the distance to the respective end point of the unit interval, the distributions of the proportions being fixed for each of the two directions. Chains of that kind were subjects of a number of studies and are of interest for some applications. Under simple broad conditions, we establis…
We prove an analog of the classical Zero-One Law for both homogeneous and nonhomogeneous Markov chains (MC). Its almost precise formulation is simple: given any event $A$ from the tail $σ$-algebra of MC $(Z_n)$, for large $n$, with probability near one, the trajectories of the MC are in states $i$, where $P(A|Z_n=i)$ is either near $0$ or near $1$. A similar statement holds for the entrance $σ$-algebra, when $n$ tends to $-\infty$. To formulate this second result, we give detailed results on the existence of nonhomogeneous Markov chains indexed by $\mathbb Z_-$ or $\mathbb Z$ in both the finit…
Density dependent Markov population processes with countably many types can often be well approximated over finite time intervals by the solution of the differential equations that describe their average drift, provided that the total population size is large. They also exhibit diffusive stochastic fluctuations on a smaller scale about this deterministic path. Here, it is shown that the individuals in such processes experience an almost deterministic environment. Small groups of individuals behave almost independently of one another, evolving as Markov jump processes, whose transition rates ar…
It is well-known that discrete-time finite-state Markov Chains, which are described by one-sided conditional probabilities which describe a dependence on the past as only dependent on the present, can also be described as one-dimensional Markov Fields, that is, nearest-neighbour Gibbs measures for finite-spin models, which are described by two-sided conditional probabilities. In such Markov Fields the time interpretation of past and future is being replaced by the space interpretation of an interior volume, surrounded by an exterior to the left and to the right. If we relax the Markov requirem…
We study the Markov chain on $\mathbf{F}_p$ obtained by applying a function $f$ and adding $\pmγ$ with equal probability. When $f$ is a linear function, this is the well-studied Chung--Diaconis--Graham process. We consider two cases: when $f$ is the extension of a rational function which is bijective, and when $f(x)=x^2$. In the latter case, the stationary distribution is not uniform and we characterize it when $p=3\pmod{4}$. In both cases, we give an almost linear bound on the mixing time, showing that the deterministic function dramatically speeds up mixing. The proofs involve establishing b…
In a probabilistic graphical model on a set of variables $V$, the Markov blanket of a random vector $B$ is the minimal set of variables conditioned to which $B$ is independent from the remaining of the variables $V \backslash B$. We generalize Markov blankets to study how a set $C$ of variables of interest depends on~$B$. Doing that, we must choose if we authorize vertices of $C$ or vertices of $V \backslash C$ in the blanket. We therefore introduce two generalizations. The Markov blanket of $B$ in $C$ is the minimal subset of $C$ conditionally to which $B$ and $C$ are independent. It is natur…
