Ergodic properties of the signal-filtering pair are studied for continuous time finite Markov chains, observed in white noise. The obtained law of large numbers is applied to the stability problem of the nonlinear filter with respect to initial conditions. The Furstenberg-Khasminskii formula is derived for the top Lyapunov exponent of the Zakai equation and is used to estimate the stability index of the filter.
We construct a stationary Markov process with trivial tail sigma-field and a nondegenerate observation process such that the corresponding nonlinear filtering process is not uniquely ergodic. This settles in the negative a conjecture of the author in the ergodic theory of nonlinear filters arising from an erroneous proof in the classic paper of H. Kunita (1971), wherein an exchange of intersection and supremum of sigma-fields is taken for granted.
Consider a filtering process associated to a hidden Markov model with densities for which both the state space and the observation space are complete, separable, metric spaces. If the underlying, hidden Markov chain is strongly ergodic and the filtering process fulfills a certain coupling condition we prove that, in the limit, the distribution of the filtering process is independent of the initial distribution of the hidden Markov chain. If furthermore the hidden Markov chain is uniformly ergodic, then we prove that the filtering process converges in distribution.
It has been established under very general conditions that the ergodic properties of Markov processes are inherited by their conditional distributions given partial information. While the existing theory provides a rather complete picture of classical filtering models, many infinite-dimensional problems are outside its scope. Far from being a technical issue, the infinite-dimensional setting gives rise to surprising phenomena and new questions in filtering theory. The aim of this paper is to discuss some elementary examples, conjectures, and general theory that arise in this setting, and to hi…
When is a nonlinear filter stable with respect to its initial condition? In spite of the recent progress, this question still lacks a complete answer in general. Currently available results indicate that stability of the filter depends on the signal ergodic properties and the observation process regularity and may fail if either of the ingredients is ignored. In this note we address the question of stability in a particular weak sense and show that the estimates of certain functions are always stable. This is verified without dealing directly with the filtering equation and turns to be inherit…
We establish moment estimates for the invariant measure of a stochastic partial differential equation describing motion by mean curvature flow in (1+1) dimension, leading to polynomial stability of the associated Markov semigroup. We also prove maximal dissipativity for the related Kolmogorov operator.
Stability problem of the Wonham filter with respect to initial conditions is addressed. The case of ergodic signals is revisited in view of a gap in the classic work of H. Kunita (1971). We give new bounds for the exponential stability rates, which do not depend on the observations. In the non-ergodic case, the stability is implied by identifiability conditions, formulated explicitly in terms of the transition intensities matrix and the observation structure.
We consider a bivariate stationary Markov chain $(X_n,Y_n)_{n\ge0}$ in a Polish state space, where only the process $(Y_n)_{n\ge0}$ is presumed to be observable. The goal of this paper is to investigate the ergodic theory and stability properties of the measure-valued process $(Π_n)_{n\ge0}$, where $Π_n$ is the conditional distribution of $X_n$ given $Y_0,...,Y_n$. We show that the ergodic and stability properties of $(Π_n)_{n\ge0}$ are inherited from the ergodicity of the unobserved process $(X_n)_{n\ge0}$ provided that the Markov chain $(X_n,Y_n)_{n\ge0}$ is nondegenerate, that is, its trans…
Exponential stability of the nonlinear filtering equation is revisited, when the signal is a finite state Markov chain. An asymptotic upper bound for the filtering error due to incorrect initial condition is derived in the case of slowly switching signal.
The problem of stability of the optimal filter is revisited. The optimal filter (or filtering process) is the conditional probability of the current state of some stochastic process (the signal process), given both present and past values of another process (the observation process). Typically the filtering process satisfies a dynamical equation, and the question investigated here concerns the stability of this dynamics. In contrast to previous work, signal processes given by the iterations of a deterministic mapping $f$ are considered, with only the initial condition being random. While the s…
We analyse the exponential stability properties of a class of measure-valued equations arising in nonlinear multi-target filtering problems. We also prove the uniform convergence properties w.r.t. the time parameter of a rather general class of stochastic filtering algorithms, including sequential Monte Carlo type models and mean eld particle interpretation models. We illustrate these results in the context of the Bernoulli and the Probability Hypothesis Density filter, yielding what seems to be the first results of this kind in this subject.
The stability and ergodicity properties of two adaptive random walk Metropolis algorithms are considered. The both algorithms adjust the scaling of the proposal distribution continuously based on the observed acceptance probability. Unlike the previously proposed forms of the algorithms, the adapted scaling parameter is not constrained within a predefined compact interval. The first algorithm is based on scale adaptation only, while the second one incorporates also covariance adaptation. A strong law of large numbers is shown to hold assuming that the target density is smooth enough and has ei…
The nonlinear filtering equation is said to be stable if it ``forgets'' the initial condition. It is known that the filter might be unstable even if the signal is an ergodic Markov chain. In general, the filtering stability requires stronger signal ergodicity provided by the, so called, mixing condition. The latter is formulated in terms of the transition probability density of the signal. The most restrictive requirement of the mixing condition is the uniform positiveness of this density. We show that it might be relaxed regardless of an observation process structure.
We establish conditions for an exponential rate of forgetting of the initial distribution of nonlinear filters in $V$-norm, path-wise along almost all observation sequences. In contrast to previous works, our results allow for unbounded test functions. The analysis is conducted in an general setup involving nonnegative kernels in a random environment which allows treatment of filters and prediction filters in a single framework. The main result is illustrated on two examples, the first showing that a total variation norm stability result obtained by Douc et al. (2009) can be extended to $V$-no…
We consider a hidden Markov model with multiplicative noise emerging from studies of software reliability. We show the stability of the optimal filter with respect to general initial conditions in the total variation- and $L^p$-norm and deduce explicit rates. Remarkably, stability turns out to be independent of the ergodic behavior of the signal.
We study the stability of single server retrial queues under general distribution for retrial times and stationary ergodic service times, for three main retrial policies studied in the literature: classical linear, constant and control policies. The approach used is the renovating events approach to obtain sufficient stability conditions by strong coupling convergence of the process modeling the dynamics of the system to a unique stationary ergodic regime. We also obtain instability conditions by convergence in distribution to improper limiting sequences.
We consider a large family of discrete and continuous time controlled Markov processes and study an ergodic risk-sensitive minimization problem. Under a blanket stability assumption, we provide a complete analysis to this problem. In particular, we establish uniqueness of the value function and verification result for optimal stationary Markov controls, in addition to the existence results. We also revisit this problem under a near-monotonicity condition but without any stability hypothesis. Our results also include policy improvement algorithms both in discrete and continuous time frameworks.
In this paper we revisit the results of Loynes (1962) on stability of queues for ergodic arrivals and services, and show examples when the arrivals are bounded and ergodic, the service rate is constant, and under stability the limit distribution has larger than exponential tail.
The ensemble Kalman filter (EnKF) and ensemble square root filter (ESRF) are data assimilation methods used to combine high dimensional, nonlinear dynamical models with observed data. Despite their widespread usage in climate science and oil reservoir simulation, very little is known about the long-time behavior of these methods and why they are effective when applied with modest ensemble sizes in large dimensional turbulent dynamical systems. By following the basic principles of energy dissipation and controllability of filters, this paper establishes a simple, systematic and rigorous framewo…
In this paper, we consider an anticipative nonlinear filtering problem, in which the observation noise is correlated with the past of the signal. This new signal-observation model has its applications in both finance models with insider trading and in engineering. We derive a new equation for the filter in this context, analyzing both the nonlinear and the linear cases. We also handle the case of a finite filter with Volterra type observation. The performance of our algorithm is presented through numerical experiments.
In this paper, we study the problem of estimating a Markov chain $X$(signal) from its noisy partial information $Y$, when the transition probability kernel depends on some unknown parameters. Our goal is to compute the conditional distribution process ${\mathbb P}\{X_n|Y_n,...,Y_1\}$, referred to hereafter as the {\it optimal filter}. Following a standard Bayesian technique, we treat the parameters as a non-dynamic component of the Markov chain. As a result, the new Markov chain is not going to be mixing, even if the original one is. We show that, under certain conditions, the optimal filters …
We study ergodic properties of a family of traffic maps acting in the space of bi-infinite sequences of real numbers. The corresponding dynamics mimics the motion of vehicles in a simple traffic flow, which explains the name. Using connections to topological Markov chains we obtain nontrivial invariant measures, prove their stochastic stability, and calculate the topological entropy. Technically these results in the deterministic setting are related to the construction of measures of maximal entropy via measures uniformly distributed on periodic points of a given period, while in the random se…
Affine jump-diffusions constitute a large class of continuous-time stochastic models that are particularly popular in finance and economics due to their analytical tractability. Methods for parameter estimation for such processes require ergodicity in order establish consistency and asymptotic normality of the associated estimators. In this paper, we develop stochastic stability conditions for affine jump-diffusions, thereby providing the needed large-sample theoretical support for estimating such processes. We establish ergodicity for such models by imposing a `strong mean reversion' conditio…
This paper is a continuation of the study on the stability speed for Markov processes. It extends the previous study of the ergodic convergence speed to the non-ergodic one, in which the processes are even allowed to be explosive or having general killings. At the beginning stage, this paper is concentrated on the birth-death processes. According to the classification of the boundaries, there are four cases plus one having general killings. In each case, some dual variational formulas for the convergence rate are presented, from which the criterion for the positivity of the rate and an approxi…
In this paper, new conditions for the stability of V-geometrically ergodic Markov chains are introduced. The results are based on an extension of the standard perturbation theory formulated by Keller and Liverani. The continuity and higher regularity properties are investigated. As an illustration, an asymptotic expansion of the invariant probability measure for an autoregressive model with i.i.d. noises (with a non-standard probability density function) is obtained.
This paper develops a connection between the asymptotic stability of nonlinear filters and a notion of observability. We consider a general class of hidden Markov models in continuous time with compact signal state space, and call such a model observable if no two initial measures of the signal process give rise to the same law of the observation process. We demonstrate that observability implies stability of the filter, i.e., the filtered estimates become insensitive to the initial measure at large times. For the special case where the signal is a finite-state Markov process and the observati…
We consider a discrete time hidden Markov model where the signal is a stationary Markov chain. When conditioned on the observations, the signal is a Markov chain in a random environment under the conditional measure. It is shown that this conditional signal is weakly ergodic when the signal is ergodic and the observations are nondegenerate. This permits a delicate exchange of the intersection and supremum of $σ$-fields, which is key for the stability of the nonlinear filter and partially resolves a long-standing gap in the proof of a result of Kunita [J. Multivariate Anal. 1 (1971) 365--393]. …
For particle filters and ensemble Kalman filters it is of practical importance to understand how and why data assimilation methods can be effective when used with a fixed small number of particles, since for many large-scale applications it is not practical to deploy algorithms close to the large particle limit asymptotic. In this paper we address this question for particle filters and, in particular, study their accuracy (in the small noise limit) and ergodicity (for noisy signal and observation) without appealing to the large particle number limit. We first overview the accuracy and minoriza…
We give elementary proof of a stability result concerning an exponential asymptotic ($t\to\infty$) for filtering estimates generated by wrongly initialized Wonham filter. This proof is based on new exponential bound having independent interest.
A hidden Markov model is called observable if distinct initial laws give rise to distinct laws of the observation process. Observability implies stability of the nonlinear filter when the signal process is tight, but this need not be the case when the signal process is unstable. This paper introduces a stronger notion of uniform observability which guarantees stability of the nonlinear filter in the absence of stability assumptions on the signal. By developing certain uniform approximation properties of convolution operators, we subsequently demonstrate that the uniform observability condition…
This thesis is concerned with the stochastic filtering problem for a hidden Markov model (HMM) with the white noise observation model. For this filtering problem, we make three types of original contributions: (1) dual controllability characterization of stochastic observability, (2) dual minimum variance optimal control formulation of the stochastic filtering problem, and (3) filter stability analysis using the dual optimal control formulation. For the first contribution of this thesis, a backward stochastic differential equation (BSDE) is proposed as the dual control system. The observabilit…
Motivated by stability questions on piecewise deterministic Markov models of bacterial chemotaxis, we study the long time behavior of a variant of the classic telegraph process having a non-constant jump rate that induces a drift towards the origin. We compute its invariant law and show exponential ergodicity, obtaining a quantitative control of the total variation distance to equilibrium at each instant of time. These results rely on an exact description of the excursions of the process away from the origin and on the explicit construction of an original coalescent coupling for both velocity …
This paper is concerned with the following problem: Given a stochastic non-linear system controlled over a noisy channel, what is the largest class of channels for which there exist coding and control policies so that the closed loop system is stochastically stable? Stochastic stability notions considered are stationarity, ergodicity or asymptotic mean stationarity. We do not restrict the state space to be compact, for example systems considered can be driven by unbounded noise. Necessary and sufficient conditions are obtained for a large class of systems and channels. A generalization of Bode…
In this paper we study the uniform stability properties of two classes of parallel server networks with multiple classes of jobs and multiple server pools of a tree topology. These include a class of networks with a single non-leaf server pool, such as the 'N' and 'M' models, and networks of any tree topology with class-dependent service rates. We show that with $\sqrt{n}$ safety staffing, and no abandonment, in the Halfin--Whitt regime, the diffusion-scaled controlled queueing processes are exponentially ergodic and their invariant probability distributions are tight, uniformly over all stati…
The stability properties of matrix-valued Riccati diffusions are investigated. The matrix-valued Riccati diffusion processes considered in this work are of interest in their own right, as a rather prototypical model of a matrix-valued quadratic stochastic process. Under rather natural observability and controllability conditions, we derive time-uniform moment and fluctuation estimates and exponential contraction inequalities. Our approach combines spectral theory with nonlinear semigroup methods and stochastic matrix calculus. This analysis seem to be the first of its kind for this class of ma…
This paper is concerned with the problem of nonlinear filter stability of ergodic Markov processes. The main contribution is the conditional Poincaré inequality (PI), which is shown to yield filter stability. The proof is based upon a recently discovered duality which is used to transform the nonlinear filtering problem into a stochastic optimal control problem for a backward stochastic differential equation (BSDE). Based on these dual formalisms, a comparison is drawn between the stochastic stability of a Markov process and the filter stability. The latter relies on the conditional PI describ…
The nonlinear filter for an ergodic signal observed in white noise is said to achieve maximal accuracy if the stationary filtering error vanishes as the signal to noise ratio diverges. We give a general characterization of the maximal accuracy property in terms of various systems theoretic notions. When the signal state space is a finite set explicit necessary and sufficient conditions are obtained, while the linear Gaussian case reduces to a classic result of Kwakernaak and Sivan (1972).
