We consider a linear stochastic differential equation with stochastic drift. We study the problem of approximating the solution of such equation through an Ornstein-Uhlenbeck type process, by using direct methods of calculus of variations. We show that general power cost functionals satisfy the conditions for existence and uniqueness of the approximation. We provide some examples of general interest and we give bounds on the goodness of the corresponding approximations. Finally, we focus on a model of a neuron embedded in a simple network and we study the approximation of its activity, by exploiting the aforementioned results.
This paper studies the last-column-block-augmented northwest-corner truncation (LC-block-augmented truncation, for short) of discrete-time block-monotone Markov chains under subgeometric drift conditions. The main result of this paper is to present an upper bound for the total variation distance between the stationary probability vectors of a block-monotone Markov chain and its LC-block-augmented truncation. The main result is extended to Markov chains that themselves may not be block monotone but are block-wise dominated by block-monotone Markov chains satisfying modified drift conditions. Fi…
A careful look at rough path topology applied to Brownian motion reveals new possible properties of the well-known Lévy area, in particular the presence of an intrinsic drift of this area. Using renormalization limit of Markov chains on periodic graphs, we present a construction of such a non-trivial drift and give an explicit formula for it. Several examples for which explicit computations are made are included.
This paper studies the augmented truncation of discrete-time block-monotone Markov chains under geometric drift conditions. We first present a bound for the total variation distance between the stationary distributions of an original Markov chain and its augmented truncation. We also obtain such error bounds for more general cases where an original Markov chain itself may not be block-monotone but is block-wise dominated by a block-monotone Markov chain. Finally we discuss the application of our results to GI/G/1-type Markov chains.
We consider a Markov chain on $R^+$ with asymptotically zero drift and finite second moments of jumps which is positive recurrent. A power-like asymptotic behaviour of the invariant tail distribution is proven; such a heavy-tailed invariant measure happens even if the jumps of the chain are bounded. Our analysis is based on test functions technique and on construction of a harmonic function.
In this paper we construct a conservative Markov semi-group with generator $L=Δ+b\cdot\nabla$ on $\mathbb{R}^n$, where $b$ is a divergence-free vector field which belongs to $L^{2}\cap L^{p}$ with $\frac{n}{2}<p$. The research is motivated by the question of understanding the blow-up solutions of the fluid dynamic equations, which attracts a lot of attention in recent years.
Let $(X_n)_{n=0}^\infty$ denote a Markov chain on a Polish space that has a stationary distribution $\varpi$. This article concerns upper bounds on the Wasserstein distance between the distribution of $X_n$ and $\varpi$. In particular, an explicit geometric bound on the distance to stationarity is derived using generalized drift and contraction conditions whose parameters vary across the state space. These new types of drift and contraction allow for sharper convergence bounds than the standard versions, whose parameters are constant. Application of the result is illustrated in the context of …
The statement of the mean field approximation theorem in the mean field theory of Markov processes particularly targets the behaviour of population processes with an unbounded number of agents. However, in most real-world engineering applications one faces the problem of analysing middle-sized systems in which the number of agents is bounded. In this paper we build on previous work in this area and introduce the mean drift. We present the concept of population processes and the conditions under which the approximation theorems apply, and then show how the mean drift is derived through a system…
Let $(X_n)$ be a Markov chain on a standard borelian space $\mathbb{X}$. Any stopping time $τ$ such that $\mathbb{E}_xτ$ is finite for all $x\in\mathbb{X}$ induces a Markov chain in $\mathbb{X}$. In this article, we show that there is a bijection between the invariant measures for the original chain and for the induced one. We then study drift functions and prove a few relations that link the Markov operator for the original chain and for the induced one. The aim is to use this drift function and the induced operator to link the solution to Poisson's equation $(I\_d-P)g=f$ for the original cha…
We consider a form of state-dependent drift condition for a general Markov chain, whereby the chain subsampled at some deterministic time satisfies a geometric Foster-Lyapunov condition. We present sufficient criteria for such a drift condition to exist, and use these to partially answer a question posed by Connor & Kendall (2007) concerning the existence of so-called 'tame' Markov chains. Furthermore, we show that our 'subsampled drift condition' implies the existence of finite moments for the return time to a small set.
We consider a sequence of Markov chains weakly convergent to a diffusion. We suppose that a drift term contains a linearly increasing component. The usual parametrix method fails because of this unbounded drift term. We show how to modify the parametrix method to obtain local limit theorems for this case.
Many applications in networked control require intermittent access of a controller to a system, as in event-triggered systems or information constrained control applications. Motivated by such applications and extending previous work on Lyapunov-theoretic drift criteria, we establish both subgeometric and geometric rates of convergence for Markov chains under state dependent random time drift criteria. We quantify how the rate of ergodicity, nature of Lyapunov functions, their drift properties, and the distributions of stopping times are related. We finally study an application in networked co…
We study the recurrence/transience phase transition for Markov chains on $\mathbb{R}_+$, $\mathbb{R}$, and $\mathbb{R}^2$ whose increments have heavy tails with exponent in $(1,2)$ and asymptotically zero mean. This is the infinite-variance analogue of the classical Lamperti problem. On $\mathbb{R}_+$, for example, we show that if the tail of the positive increments is about $c y^{-α}$ for an exponent $α\in (1,2)$ and if the drift at $x$ is about $b x^{-γ}$, then the critical regime has $γ= α-1$ and recurrence/transience is determined by the sign of $b + cπ\textrm{cosec} (πα)$. On $\mathbb{R}$…
We prove a full large deviations principle in large time, for a diffusion process with random drift V, which is a centered Gaussian shear flow random field. The large deviations principle is established in a ``quenched'' setting, i.e. is valid almost surely in the randomness of V.
By appealing to renewal theory we determine the equations that the mean exit time of a continuous-time random walk with drift satisfies both when the present coincides with a jump instant or when it does not. Particular attention is paid to the corrections ensuing from the non-Markovian nature of the process. We show that when drift and jumps have the same sign the relevant integral equations can be solved in closed form. The case when holding times have the classical Erlang distribution is considered in detail.
The convergence to the stationary regime is studied for Stochastic Differential Equations driven by an additive Gaussian noise and evolving in a semi-contractive environment, i.e. when the drift is only contractive out of a compact set but does not have repulsive regions. In this setting, we develop a synchronous coupling strategy to obtain sub-exponential bounds on the rate of convergence to equilibrium in Wasserstein distance. Then by a coalescent coupling close to terminal time, we derive a similar bound in total variation distance.
We solve the Skorokhod embedding problem for a class of Gaussian processes including Brownian motion with non-linear drift. Our approach relies on solving an associated strongly coupled system of Forward Backward Stochastic Differential Equation (FBSDE), and investigating the regularity of the obtained solution. For this purpose we extend the existence, uniqueness and regularity theory of so called decoupling fields for Markovian FBSDE to a setting in which the coefficients are only locally Lipschitz continuous.
Motivated by applications to proving regularity of solutions to degenerate parabolic equations arising in population genetics, we study existence, uniqueness and the strong Markov property of weak solutions to a class of degenerate stochastic differential equations. The stochastic differential equations considered in our article admit solutions supported in the set $[0,\infty)^n\times\mathbb{R}^m$, and they are degenerate in the sense that the diffusion matrix is not strictly elliptic, as the smallest eigenvalue converges to zero proportional to the distance to the boundary of the domain, and …
We consider a multidimensional SDE with a Gaussian noise and a drift vector being a vector function of bounded variation. We prove the existence of generalized derivative of the solution with respect to the initial conditions and represent the derivative as a solution of a linear SDE with coefficients depending on the initial process. The representation obtained is a natural generalization of the expression for the derivative in the smooth case. The theory of continuous additive functionals is used.
Random walks on $\mathbb{Z}$ with metastable Gaussian distribution caused by linear drift with application to the contact process on the complete graph
This review paper provides an introduction of Markov chains and their convergence rates which is an important and interesting mathematical topic which also has important applications for very widely used Markov chain Monte Carlo (MCMC) algorithm. We first discuss eigenvalue analysis for Markov chains on finite state spaces. Then, using the coupling construction, we prove two quantitative bounds based on minorization condition and drift conditions, and provide descriptive and intuitive examples to showcase how these theorems can be implemented in practice. This paper is meant to provide a gener…
We present a class of Gauss-Markov processes which can be represented as space-time scaled stationary Ornstein-Uhlenbeck processes defined on the real line. We give several explicit examples of the representation for certain Gauss bridge processes. As an application, we derive a formula for the density function of the supremum location of certain standardized Gauss-Markov processes on compact time intervals. We also present some sufficient conditions under which mean centered Gauss-Markov processes take zero at a fixed time with probability one.
The paper deals with the regression model $X_t = θt + B_t$, $t\in[0, T ]$, where $B=\{B_t, t\geq 0\}$ is a centered Gaussian process with stationary increments. We study the estimation of the unknown parameter $θ$ and establish the formula for the likelihood function in terms of a solution to an integral equation. Then we find the maximum likelihood estimator and prove its strong consistency. The results obtained generalize the known results for fractional and mixed fractional Brownian motion.
We propose a new classification scheme for diffusion processes for which the backward Kolmogorov equation is solvable in analytically closed form by reduction to hypergeometric equations of the Gaussian or confluent type. The construction makes use of transformations of diffusion processes to eliminate the drift which combine a measure change given by Doob's h-transform and a diffeomorphism. Such transformations have the important property of preserving analytic solvability of the process: the transition probability density for the driftless process can be expressed through the transition prob…
The Laplace transform of partial sums of the square of a non-centered Gauss-Markov process, conditioning on its starting point, is explicitly computed. The parameters of multiplicative ergodicity are deduced.
K. Itô characterised in \cite{ito} zero-mean stationary Gauss Markov-processes evolving on a class of infinite-dimensional spaces. In this work we extend the work of Itô in the case of Hilbert spaces: Gauss-Markov families that are time-homogenous are identified as solutions to linear stochastic differential equations with singular coefficients. Choosing an appropriate locally convex topology on the space of weakly sequentially continuous functions we also characterize the transition semigroup, the generator and its core thus providing an infinite-dimensional extension of the classical result …
In this article, we derive the explicit transition density functions of skew Brownian motion (SBM in abbreviation) with two-valued drift for all $t>0$. As an important step of this result, it is also shown in this paper that SBM with two-valued drift is a strong Markov process by finding its symmetrizing measure and canonical scale function, from which one can tell what values of the drift make such a process transient or recurrent.
We find a representation of the integral of a Gauss-Markov process in the interval [0, t], in terms of Brownian motion. Moreover, some connections with first-passagetime problems are discussed, and some examples are reported.
In this paper, we establish a probabilistic representation as well as some integration by parts formulae for the marginal law at a given time maturity of some stochastic volatility model with unbounded drift. Relying on a perturbation technique for Markov semigroups, our formulae are based on a simple Markov chain evolving on a random time grid for which we develop a tailor-made Malliavin calculus. Among other applications, an unbiased Monte Carlo path simulation method stems from our formulas so that it can be used in order to numerically compute with optimal complexity option prices as well …
We consider a multidimensional stochastic differential equation with a Gaussian noise and a drift vector having a jump discontinuity along a hyperplane. The large time behavior of the distance between two solutions starting from different points is studied.We consider a multidimensional stochastic differential equation with a Gaussian noise and a drift vector having a jump discontinuity along a hyperplane. The large time behavior of the distance between two solutions starting from different points is studied.
We consider a time-homogeneous Markov chain $X_n$, $n\ge0$, valued in ${\bf R}$. Suppose that this chain is transient, that is, $X_n$ generates a $σ$-finite renewal measure. We prove the key renewal theorem under condition that this chain has asymptotically homogeneous at infinity jumps and asymptotically positive drift.
We find the class, ${\cal{C}}_k, k \ge 0$, of all zero mean stationary Gaussian processes, $Y(t), ~t \in \reals$ with $k$ derivatives, for which \begin{equation} Z(t) \equiv (Y^{(0)}(t), Y^{(1)}(t), \ldots, Y^{(k)}(t) ), ~ t \ge 0 \end{equation} \noindent is a $(k+1)$-vector Markov process. (here, $Y^{(0)}(t) = Y(t)$).
We prove a version of the multidimensional Fourth Moment Theorem for chaotic random vectors, in the general context of diffusion Markov generators. In addition to the usual componentwise convergence and unlike the infinite-dimensional Ornstein-Uhlenbeck generator case, another moment-type condition is required to imply joint convergence of of a given sequence of vectors.
The study of multidimensional stochastic processes involves complex computations in intricate functional spaces. In particular, the diffusion processes, which include the practically important Gauss-Markov processes, are ordinarily defined through the theory of stochastic integration. Here, inspired by the Lévy-Cieselski construction of the Wiener process, we propose an alternative representation of multidimensional Gauss-Markov processes as expansions on well-chosen Schauder bases, with independent random coefficients of normal law with zero mean and unitary variance. We thereby offer a natur…
Bacteria are known to exchange genetic information by horizontal gene transfer. Since the frequency of homologous recombination depends on the similarity of recombining segments, several studies examined whether this could lead to the emergence of subspecies. Most of them used simulations of fixed size Wright-Fisher populations, in which the genetic drift should be taken into account. Here, we use non-linear Markov chains to describe a bacterial population evolving under mutation and recombination processes. We consider a population as a space of genomes and a probability measure on it. Thus, …
Let $u=\{u(t,x);t \in [0,T], x \in {\mathbb{R}}^{d}\}$ be the process solution of the stochastic heat equation $u_{t}=Δu+ \dot F, u(0,\cdot)=0$ driven by a Gaussian noise $\dot F$, which is white in time and has spatial covariance induced by the kernel $f$. In this paper we prove that the process $u$ is locally germ Markov, if $f$ is the Bessel kernel of order $α=2k,k \in \bN_{+}$, or $f$ is the Riesz kernel of order $α=4k,k \in \bN_{+}$.
This paper is a natural continuation of [8], where strong Markov processes are constructed in time inhomogeneous setting with Borel measurable uniformly bounded and uniformly nondegenerate diffusion and drift in $L_{d+1}(\mathbb{R}^{d+1})$. Here we study some properties of these processes such as higher summability of Green's functions, boundedness of resolvent operators in Lebesgue spaces, establish Itô's formula, and so on.
In this paper we consider the nonparametric functional estimation of the drift of Gaussian processes using Paley-Wiener and Karhunen-Loève expansions. We construct efficient estimators for the drift of such processes, and prove their minimaxity using Bayes estimators. We also construct superefficient estimators of Stein type for such drifts using the Malliavin integration by parts formula and stochastic analysis on Gaussian space, in which superharmonic functionals of the process paths play a particular role. Our results are illustrated by numerical simulations and extend the construction of J…
The solution to a multivariate linear Stochastic Differential Equation (SDE) with constant initial state is well known to be a Gaussian Markov process, but its covariance kernel involves the solution to an integral equation in the general case. We show that the covariance kernel has a simpler semi-parametric form for families of such solutions representing increments of a common process. We also show that a covariance kernel of a particular parametric form is necessary and sufficient for a solution to possess stationary increments and for a Gaussian process, in considerable generality, to have…
We describe stochastic calculus in the context of processes that are driven by an adapted point process of locally finite intensity and are differentiable between jumps. This includes Markov chains as well as non-Markov processes. By analogy with Itô processes we define the drift and diffusivity, which we then use to describe a general sample path estimate. We then give several examples, including ODE approximation, processes with linear drift, first passage times, and an application to the stochastic logistic model.
We study reflecting Brownian motion with drift constrained to a wedge in the plane. Our first set of results provide necessary and sufficient conditions for existence and uniqueness of a solution to the corresponding submartingale problem with drift, and show that its solution possesses the Markov and Feller properties. Next, we study a version of the problem with absorption at the vertex of the wedge. In this case, we provide a condition for existence and uniqueness of a solution to the problem and some results on the probability of the vertex being reached.
The "drift-and-minorization" method, introduced and popularized in (Rosenthal, 1995; Meyn and Tweedie, 1994; Meyn and Tweedie, 2012), remains the most popular approach for bounding the convergence rates of Markov chains used in statistical computation. This approach requires estimates of two quantities: the rate at which a single copy of the Markov chain "drifts" towards a fixed "small set", and a "minorization condition" which gives the worst-case time for two Markov chains started within the small set to couple with moderately large probability. In this paper, we build on (Oliveira, 2012; Pe…
We consider a time inhomogeneous strong Markov process $(ξ_t)_{t\ge 0}$ taking values in a Polish state space whose semigroup has a $T$-periodic structure. We give simple conditions which imply ergodicity of the grid chain $(ξ_{kT})_{k\in \mathbb{N}_0}$ In case of $d$-dimensional possibly degenerate diffusions, the conditions are stated in terms of drift and diffusion coefficient of the process.
We consider an optimal control problem, where a Brownian motion with drift is sequentially observed, and the sign of the drift coefficient changes at jump times of a symmetric two-state Markov process. The Markov process itself is not observable, and the problem consist in finding a {-1,1}-valued process that tracks the unobservable process as close as possible. We present an explicit construction of such a process.
We propose a new tamed Milstein-type scheme for stochastic differential equation with Markovian switching when drift coefficient is assumed to grow super-linearly. The strong rate of convergence is shown to be equal to $1.0$ under mild regularity (e.g. once differentiability) requirements on drift and diffusion coefficients. Novel techniques are developed to tackle two-fold difficulties arising due to jumps of the Markov chain and the reduction of regularity requirements on the coefficients.
Over the last three decades, there has been a considerable effort within the applied probability community to develop techniques for bounding the convergence rates of general state space Markov chains. Most of these results assume the existence of drift and minorization (d\&m) conditions. It has often been observed that convergence rate bounds based on single-step d\&m tend to be overly conservative, especially in high-dimensional situations. This article builds a framework for studying this phenomenon. It is shown that any convergence rate bound based on a set of d\&m conditions cannot do bet…
