Stochastic chains with memory of variable length constitute an interesting family of stochastic chains of infinite order on a finite alphabet. The idea is that for each past, only a finite suffix of the past, called context, is enough to predict the next symbol. These models were first introduced in the information theory literature by Rissanen (1983) as a universal tool to perform data compression. Recently, they have been used to model up scientific data in areas as different as biology, linguistics and music. This paper presents a personal introductory guide to this class of models focusing on the algorithm Context and its rate of convergence.
We show how to construct a topological Markov map of the interval whose invariant probability measure is the stationary law of a given stochastic chain of infinite order. In particular we caracterize the maps corresponding to stochastic chains with memory of variable length. The problem treated here is the converse of the classical construction of the Gibbs formalism for Markov expanding maps of the interval.
Here we obtain the exact asymptotics for large and moderate deviations, strong law of large numbers and central limit theorem for chains with unbounded variable length memory.
We define fully coupled forward-backward stochastic differential equations on spaces related to continuous time, finite state Markov Chains. Existence and uniqueness results of the fully coupled forward-backward stochastic differential equations on Markov Chains are obtained.
We consider binary infinite order stochastic chains perturbed by a random noise. This means that at each time step, the value assumed by the chain can be randomly and independently flipped with a small fixed probability. We show that the transition probabilities of the perturbed chain are uniformly close to the corresponding transition probabilities of the original chain. As a consequence, in the case of stochastic chains with unbounded but otherwise finite variable length memory, we show that it is possible to recover the context tree of the original chain, using a suitable version of the alg…
We present a new perfect simulation algorithm for stationary chains having unbounded variable length memory. This is the class of infnite memory chains for which the family of transition probabilities is represented by a probabilistic context tree. We do not assume any continuity condition: our condition is expressed in terms of the structure of the context tree. More precisely, the length of the contexts is a deterministic function of the distance to the last occurrence of some determined string of symbols. It turns out that the resulting class of chains can be seen as a natural extension of …
In this paper, we study inference for chains of variable order under two distinct contamination regimes. Consider we have a chain of variable memory on a finite alphabet containing zero. At each instant of time an independent coin is flipped and if it turns head a contamination occurs. In the first regime a zero is read independent of the value of the chain. In the second regime, the value of another chain of variable memory is observed instead of the original one. Our results state that the difference between the transition probabilities of the original process and the corresponding ones of t…
We study ergodic properties of nonlinear Markov chains and stochastic McKean-Vlasov equations. For nonlinear Markov chains we obtain sufficient conditions for existence and uniqueness of an invariant measure and uniform ergodicity. We also prove optimality of these conditions. For stochastic McKean-Vlasov equations we establish exponential convergence of their solutions to stationarity in the total variation metric under Veretennikov-Khasminskii-type conditions.
We present an alternative proof for the existence of solutions of stochastic functional differential equations satisfying a global Lipschitz condition. The proof is based on an approximation scheme in which the continuous path dependence does not go up to the present: there is a memory gap. Strong convergence is obtained by closing the gap. Such approximation is particularly useful when extending stochastic models with discrete delay to models with continuous full finite memory.
In this paper, we deal with a class of mean-field backward stochastic differential equations (BSDEs) related to finite state, continuous time Markov chains. We obtain the existence and uniqueness theorem and a comparison theorem for solutions of one-dimensional mean-field BSDEs under Lipschitz condition.
In this paper, we provide an estimate for the solutions of reflected backward stochastic differential equations (RBSDEs) driven by a Markov chain, derive a continuous dependence property for their solutions with respect to the parameters of the equations, and show similar properties for solutions of backward stochastic differential equations (BSDEs). We finally establish a comparison result for the solutions of RBSDEs driven by a Markov chain.
We introduce an statistical mechanical formalism for the study of discrete-time stochastic processes with which we prove: (i) General properties of extremal chains, including triviality on the tail $σ$-algebra, short-range correlations, realization via infinite-volume limits and ergodicity. (ii) Two new sufficient conditions for the uniqueness of the consistent chain. The first one is a transcription of a criterion due to Georgii for one-dimensional Gibbs measures, and the second one corresponds to Dobrushin criterion in statistical mechanics. (iii) Results on loss of memory and mixing propert…
In this paper we study various properties of finite stochastic systems or hidden Markov chains as they are alternatively called. We discuss their construction following different approaches and we also derive recursive filtering formulas for the different systems that we consider. The key tool is a simple lemma on conditional expectations.
The asymptotic log-Harnack inequality is established for several different models of stochastic differential systems with infinite memory: non-degenerate SDEs, Neutral SDEs, semi-linear SPDEs, and stochastic Hamiltonian systems. As applications, the following properties are derived for the associated segment Markov semigroups: asymptotic heat kernel estimate; uniqueness of the invariant probability measure; asymptotic gradient estimate and hence, asymptotically strong Feller property; and asymptotic irreducibilty.
In 2013, Lu and Ren \cite {luren} considered anticipated backward stochastic differential equations driven by finite state, continuous time Markov chain noise and established the existence and uniqueness of the solutions of these equations and a scalar comparison theorem. In this paper, we provide an estimate for their solutions and study the duality between these equations and stochastic differential delayed equations with Markov chain noise. Finally we derive another comparison theorem for these solutions depending only on the two drivers.
We study linear-quadratic stochastic differential games on directed chains inspired by the directed chain stochastic differential equations introduced by Detering, Fouque, and Ichiba. We solve explicitly for Nash equilibria with a finite number of players and we study more general finite-player games with a mixture of both directed chain interaction and mean field interaction. We investigate and compare the corresponding games in the limit when the number of players tends to infinity. The limit is characterized by Catalan functions and the dynamics under equilibrium is an infinite-dimensional …
Assume we have two stochastic chains taking values in a finite alphabet. These chains may be of infinite order. Assume also that these chains are coupled in such a way that given the past of both chains they have a not too large probability of differing. This is the case when we observe a chain through a noisy channel. This situation presumably also occurs in models for the brain activity when a chain of stimuli is presented to a volunteer and we observe a corresponding chain of neurophysiological recordings. The question is how these two chains are quantitatively related. Under suitable condi…
In this paper we discuss backward stochastic differential equations with Markov chain noise, having continuous drivers. We obtain the existence of a solution which is possibly not unique. Moreover, we show there is a minimal solution for this kind of equation and derive the corresponding comparison result. This is applied to pricing of European options in a market with Markov chain noise.
We consider backward stochastic differential equations (BSDEs) related to finite state, continuous time Markov chains. We show that appropriate solutions exist for arbitrary terminal conditions, and are unique up to sets of measure zero. We do not require the generating functions to be monotonic, instead using only an appropriate Lipschitz continuity condition.
In this paper, we consider stationarity of a class of second-order stochastic evolution equations with memory, driven by Wiener processes or Levy jump processes, in Hilbert spaces. The strategy is to formulate by reduction some first-order systems in connection with the stochastic equations under investigation. We develop asymptotic behavior of dissipative second-order equations and then apply them to time delay systems through Gearhart-Pruss-Greiner's theorem. The stationary distribution of the system under consideration is the projection on the first coordinate of the corresponding stationar…
For a heat equation with memory driven by a Lévy-type noise we establish the existence of a unique solution. The main part of the article focuses on the Freidlin-Wentzell large deviation principle of the solutions of heat equation with memory driven by a Lévy-type noise. For this purpose, we exploit the recently introduced weak convergence approach.
We study the smoothness of the solution of the directed chain stochastic differential equations, where each process is affected by its neighborhood process in an infinite directed chain graph, introduced by Detering et al. (2020). Because of the auxiliary process in the chain-like structure, classic methods of Malliavin derivatives are not directly applicable. Namely, we cannot make a connection between the Malliavin derivative and the first order derivative of the state process. It turns out that the partial Malliavin derivatives can be used here to fix this problem.
Stemmed from the derivation of the optimal control to a stochastic linear-quadratic control problem with Markov jumps, we study one kind of backward stochastic differential equations (BSDEs) that the generator f is affected by a Markovian switching. Then, the case that the Markov chain is involved in a large state space is considered. Following the classical approach, a hierarchical approach is adopted to reduce the complexity and a singularly perturbed Markov chain is involved. We will study the asymptotic property of BSDE with the singularly perturbed Markov chain. At last, as an application…
Comparison and converse comparison theorems are important parts of the research on backward stochastic differential equations. In this paper, we obtain comparison results for one dimensional backward stochastic differential equations with Markov chain noise, extending and generalizing previous work under natural and simplified hypotheses, and establish a converse comparison theorem for the same type of equation after giving the definition and properties of a type of nonlinear expectation: $f$-expectation.
We present two main theorems for stochastic processes with a Markov regime-switching model. First, we work on an existence-uniqueness theorem for a Stochastic Differential Delay Equation with Jumps and Regimes (SDDEJRs). Then, we provide the duality between an SDDEJR and an Anticipated Backward Stochastic Differential Equation with Jumps and Regimes (ABSDEJRs). Our goal is to provide two technical and basis theorems for the future theoretical and applied developments of time-delayed and time-advanced models.
In this paper we show the convergence of the long-term return $t^{-μ}\int_0^tX(s)\d s$ for some $μ\geq1$, where $X$ is the short-term interest rate which follows an extension of Cox-Ingersoll-Ross type model with jumps and memory, and, as an application, we also investigate the corresponding behavior of two-factor Cox-Ingersoll-Ross model with jumps and memory
Our aim is to find sufficient conditions for weak convergence of stochastic integrals with respect to the state occupation measure of a Markov chain. First, we study properties of the state indicator function and the state occupation measure of a Markov chain. In particular, we establish weak convergence of the state occupation measure under a scaling of the generator matrix. Then, relying on the connection between the state occupation measure and the Dynkin martingale related to the state indicator function, we provide sufficient conditions for weak convergence of stochastic integrals with re…
We derive sufficient and necessary optimality conditions in terms of a stochastic maximum principle (SMP) for controls associated with cost functionals of mean-field type, under dynamics driven by a class of Markov chains of mean-field type which are pure jump processes obtained as solutions of a well-posed martingale problem. As an illustration, we apply the result to generic examples of control problems as well as some applications.
We develop some sufficient conditions for the stochastic ordering between hitting times, in a fixed state, for two Markov chains. In particular, we focus attention on the so called \emph{skip-free} case. In the analysis of such a case, we develop a special type of coupling. We also compare different types of relations between two, non-necessarily skip-free, Markov chains on the same state space. Such relations have a natural role in establishing the usual and the asymptotic stochastic ordering between the probability distributions of hitting times. Finally, we present some discussions and exam…
In this paper, we introduce a new kind of reflected backward stochastic differential equations (RBSDEs) driven by a martingale, in a Markov chain model, but not driven by Brownian motion, and give existence and uniqueness results for the new equations. Then we discuss American options in a finite state Markov chain model, in the presence of a stochastic discount function (SDF) and using the theory of the new RBSDEs. We show that there exists a constrained super-hedging strategy for an American option, which is unique in our framework as the solution to an RBSDE.
We propose a particle system of diffusion processes coupled through a chain-like network structure described by an infinite-dimensional, nonlinear stochastic differential equation of McKean-Vlasov type. It has both (i) a local chain interaction and (ii) a mean-field interaction. It can be approximated by a limit of finite particle systems, as the number of particles goes to infinity. Due to the local chain interaction, propagation of chaos does not necessarily hold. Furthermore, we exhibit a dichotomy of presence or absence of mean-field interaction, and we discuss the problem of detecting its…
We consider a Lotka-Volterra food chain model with possibly intra-specific competition in a stochastic environment represented by stochastic differential equations. In the non-degenerate setting, this model has already been studied by A. Hening and D. Nguyen. They provided conditions for stochastic persistence and extinction. In this paper, we extend their results to the degenerate situation in which the top or the bottom species is subject to random perturbations. Under the persistence condition, there exists a unique invariant probability measure supported by the interior of $\mathbb{R}_+^n$…
An algorithm for estimating quasi-stationary distribution of finite state space Markov chains has been proven in a previous paper. Now this paper proves a similar algorithm that works for general state space Markov chains under very general assumptions.
This paper is composed of two main results concerning chains of infinite order which are not necessarily continuous. The first one is a decomposition of the transition probability kernel as a countable mixture of unbounded probabilistic context trees. This decomposition is used to design a simulation algorithm which works as a combination of the algorithms given by Comets et al. (2002) and Gallo (2009). The second main result gives sufficient conditions on the kernel for this algorithm to stop after an almost surely finite number of steps. Direct consequences of this last result are existence …
It is well-known that there always exists at least one stationary measure compatible with a continuous g-function g. Here we prove that if the set of discontinuities of the g-function g has null measure under a candidate measure obtained by some asymptotic procedure, then this candidate measure is compatible with g. We explore several implications of this result, and discuss comparisons with the literature concerning assumptions and examples. Important part of the paper is concerned with the case of variable length memory chains, for which we obtain existence, uniqueness and weak-Bernoullicity…
In this work, we consider the regularity property of stochastic convolutions for a class of abstract linear stochastic retarded functional differential equations with unbounded operator coefficients. We first establish some useful estimates on fundamental solutions which are time delay versions of those on $C_0$-semigroups. To this end, we develop a scheme of constructing the resolvent operators for the integrodifferential equations of Volterra type since the equation under investigation is of this type in each subinterval describing the segment of its solution. Then we apply these estimates t…
We consider a class of semi-linear differential Volterra equations with memory terms, polynomial nonlinearities and random perturbation. For a broad class of nonlinearities, we study statistically steady states of the system and find that they possess regularities compatible with those of the weak solutions. Moreover, if sufficiently many directions in the phase space are stochastically forced, we employ the \emph{generalized coupling} approach to establish the existence and uniqueness of the invariant probability measure to which the system is exponentially attractive. This extends ergodicity…
We revisit the classical problem of approximating a stochastic differential equation by a discrete-time and discrete-space Markov chain. Our construction iterates Caratheodory's theorem over time to match the moments of the increments locally. This allows to construct a Markov chain with a sparse transition matrix where the number of attainable states grows at most polynomially as time increases. Moreover, the MC evolves on a tree whose nodes lie on a "universal lattice" in the sense that an arbitrary number of different SDEs can be approximated on the same tree. The construction is not tailor…
Variable Length Memory Chains (VLMC), which are generalizations of finite order Markov chains, turn out to be an essential tool to modelize random sequences in many domains, as well as an interesting object in contemporary probability theory. The question of the existence of stationary probability measures leads us to introduce a key combinatorial structure for words produced by a VLMC: the Longest Internal Suffix. This notion allows us to state a necessary and sufficient condition for a general VLMC to admit a unique invariant probability measure. This condition turns out to get a much simple…
Stochastic systems with memory naturally appear in life science, economy, and finance. We take the modelling point of view of stochastic functional delay equations and we study these structures when the driving noises admit jumps. Our results concern existence and uniqueness of strong solutions, estimates for the moments and the fundamental tools of calculus, such as the Itô formula. We study the robustness of the solution to the change of noises. Specifically, we consider the noises with infinite activity jumps versus an adequately corrected Gaussian noise. The study is presented in two diffe…
Using the regenerative scheme of Comets, Fernández and Ferrari (2002), we establish a functional central limit theorem (FCLT) for discrete time stochastic processes (chains) with summable memory decay. Furthermore, under stronger assumptions on the memory decay, we identify the limiting variance in terms of the process only. As applications, we define classes of binary autoregressive processes and power-law Ising chains for which the FCLT is fulfilled.
In this paper we give simple sufficient conditions for linear type processes with short memory that imply the invariance principle. Various examples including projective criterion are considered as applications. In particular, we treat the weak invariance principle for partial sums of linear processes with short memory. We prove that whenever the partial sums of innovations satisfy the $L_p$--invariance principle, then so does the partial sums of its corresponding linear process.
We prove that uniqueness of the stationary chain, or equivalently, of the $g$-measure, compatible with an attractive regular probability kernel is equivalent to either one of the following two assertions for this chain: (1) it is a finitary coding of an i.i.d. process with countable alphabet, (2) the concentration of measure holds at exponential rate. We show in particular that if a stationary chain is uniquely defined by a kernel that is continuous and attractive, then this chain can be sampled using a coupling-from-the-past algorithm. For the original Bramson-Kalikow model we further prove t…
We consider long-time behavior of dynamical systems perturbed by a small noise. Under certain conditions, a slow component of such a motion, which is most important for long- time evolution, can be described as a motion on the cone of invariant measures of the non- perturbed system. The case of a finite number of extreme points of the cone is considered in this paper. As is known, in the generic case, the long-time evolution can be described by a hierarchy of cycles defined by the action functional for corresponding stochastic processes. This, in particular, allows to study metastable distribu…
The exponential stability, in both mean square and almost sure senses, for energy solutions to a class of nonlinear and non-autonomous stochastic PDEs with finite memory is investigated. Various criteria for stability are obtained. An example is presented to demonstrate the main results.
A stochastic differential equation with infinite memory is considered. The drift coefficient of the equation is a nonlinear functional of the past history of the solution. Sufficient conditions for existence and uniqueness of stationary solution are given.
By starting from the stochastic Schrödinger equation and quantum trajectory theory, we introduce memory effects by considering stochastic adapted coefficients. As an example of a natural non-Markovian extension of the theory of white noise quantum trajectories we use an Ornstein-Uhlenbeck coloured noise as the output driving process. Under certain conditions a random Hamiltonian evolution is recovered. Moreover, we show that our non-Markovian stochastic Schrödinger equations unravel some master equations with memory kernels.
The mathematical model of a linear system with the short memory about own stochastic behavior is proposed. It is assumed that the system is under a continual influence of independent stochastic impulses. In a short memory approximation the expression of the stochastic process is found. An application of the model proposed to capital market processes is examined. The approach allows form a stochastic differential for processes concerned. The analog of the Black-Scholes equation for assets dealt on a market with the memory is expressed.
Population dynamics are often affected by sudden environmental perturbations. Parameters of stochastic models are often imprecise due to various uncertainties. In this paper, we formulate a stochastic Lotka Volterra food chain chemostat model that includes Levy jumps and interval parameters. Firstly, we prove the existence and uniqueness of the positive solution. Moreover, the threshold between extinction and persistence in the mean of the microorganism is obtained. Finally, some simulations are carried out to demonstrate our theoretical results.
In this paper we introduce a model which provides a new approach to the phenomenon of stochastic resonance. It is based on the study of the properties of the stationary distribution of the underlying stochastic process. We derive the formula for the spectral power amplification coefficient, study its asymptotic properties and dependence on parameters.
The initial time-dependence of a state in circumstances where it makes transitions to, or decay to, a second state has been investigated. In classical stochastic processes, the observed time dependence of transition or decay proportional to $t^2$ is attributed to the noise with memory. In contrast to quantum mechanics, the quadratic form of initial decay is unable to decelerate the evolution of the system.
To overcome topological constraints and improve the expressiveness of normalizing flow architectures, Wu, Köhler and Noé introduced stochastic normalizing flows which combine deterministic, learnable flow transformations with stochastic sampling methods. In this paper, we consider stochastic normalizing flows from a Markov chain point of view. In particular, we replace transition densities by general Markov kernels and establish proofs via Radon-Nikodym derivatives which allows to incorporate distributions without densities in a sound way. Further, we generalize the results for sampling from p…
We consider the supOU stochastic volatility model which is able to exhibit long-range dependence. For this model we give conditions for the discounted stock price to be a martingale, calculate the characteristic function, give a strip where it is analytic and discuss the use of Fourier pricing techniques. Finally, we present a concrete specification with polynomially decaying autocorrelations and calibrate it to observed market prices of plain vanilla options.
