In this paper we consider the continuous--time nonlinear filtering problem, which has an infinite--dimensional solution in general, as proved by Chaleyat--Maurel and Michel. There are few examples of nonlinear systems for which the optimal filter is finite dimensional, in particular Kalman's, Benes', and Daum's filters. In the present paper, we construct new classes of scalar nonlinear filtering problems admitting finite--dimensional filters. We consider a given (nonlinear) diffusion coefficient for the state equation, a given (nonlinear) observation function, and a given finite--dimensional exponential family of probability densities. We construct a drift for the state equation such that the resulting nonlinear filtering problem admits a finite--dimensional filter evolving in the prescribed exponential family augmented by the observaton function and its square.
We prove that the exponential distribution is the only one which satisfies a regression identity. This identity involves conditional expectation of the sample mean of record values given two record values outside of the sample.
The model consists of a signal process $X$ which is a general Brownian diffusion process and an observation process $Y$, also a diffusion process, which is supposed to be correlated to the signal process. We suppose that the process $Y$ is observed from time 0 to $s>0$ at discrete times and aim to estimate, conditionally on these observations, the probability that the non-observed process $X$ crosses a fixed barrier after a given time $t>s$. We formulate this problem as a usual nonlinear filtering problem and use optimal quantization and Monte Carlo simulations techniques to estimate the invol…
The explicit solution of the discrete time filtering problems with exponential criteria for a general Gaussian signal is obtained through an approach based on a conditional Cameron-Martin type formula. This key formula is derived for conditional expectations of exponentials of some quadratic forms of Gaussian sequences. The formula involves conditional expectations and conditional covariances in some auxiliary optimal risk-neutral filtering problem which is used in the proof. Closed form recursions of Volterra type for these ingredients are provided. Particular cases for which the results can …
This paper is concerned with the problem of nonlinear filter stability of ergodic Markov processes. The main contribution is the conditional Poincaré inequality (PI), which is shown to yield filter stability. The proof is based upon a recently discovered duality which is used to transform the nonlinear filtering problem into a stochastic optimal control problem for a backward stochastic differential equation (BSDE). Based on these dual formalisms, a comparison is drawn between the stochastic stability of a Markov process and the filter stability. The latter relies on the conditional PI describ…
The exponential stability and the concentration properties of a class of extended Kalman-Bucy filters are analyzed. New estimation concentration inequalities around partially observed signals are derived in terms of the stability properties of the filters. These non asymptotic exponential inequalities allow to design confidence interval type estimates in terms of the filter forgetting properties with respect to erroneous initial conditions. For uniformly stable signals, we also provide explicit non-asymptotic estimates for the exponential forgetting rate of the filters and the associated stoch…
We deduce the asymptotic error distribution of the Euler method for the nonlinear filtering problem with continuous-time observations. Previous works by several authors have shown that the error structure of the method is characterized by conditional expectations of some functionals of multiple stochastic integrals. Our main result is a proof of the stable convergence of a sequence of such conditional expectations, using the technique of martingale limit theorems in the spirit of Jacod.
We take advantage of recent and new results on optimal quantization theory to improve the quadratic optimal quantization error bounds for backward stochastic differential equations (BSDE) and nonlinear filtering problems. For both problems, a first improvement relies on a Pythagoras like Theorem for quantized conditional expectation. While allowing for some locally Lipschitz functions conditional densities in nonlinear filtering, the analysis of the error brings into playing a new robustness result about optimal quantizers, the so-called distortion mismatch property: $L^r$-quadratic optimal qu…
Filter stability is a classical problem in the study of partially observed Markov processes (POMP), also known as hidden Markov models (HMM). For a POMP, an incorrectly initialized non-linear filter is said to be (asymptotically) stable if the filter eventually corrects itself as more measurements are collected. Filter stability results in the literature that provide rates of convergence typically rely on very restrictive mixing conditions on the transition kernel and measurement kernel pair, and do not consider their effects independently. In this paper, we introduce an alternative approach u…
A non-Euclidean generalization of conditional expectation is introduced and characterized as the minimizer of expected intrinsic squared-distance from a manifold-valued target. The computational tractable formulation expresses the non-convex optimization problem as transformations of Euclidean conditional expectation. This gives computationally tractable filtering equations for the dynamics of the intrinsic conditional expectation of a manifold-valued signal and is used to obtain accurate numerical forecasts of efficient portfolios by incorporating their geometric structure into the estimates.
The conditional particle filter (CPF) is a promising algorithm for general hidden Markov model smoothing. Empirical evidence suggests that the variant of CPF with backward sampling (CBPF) performs well even with long time series. Previous theoretical results have not been able to demonstrate the improvement brought by backward sampling, whereas we provide rates showing that CBPF can remain effective with a fixed number of particles independent of the time horizon. Our result is based on analysis of a new coupling of two CBPFs, the coupled conditional backward sampling particle filter (CCBPF). …
In this paper, we consider the extreme behavior of a Gaussian random field $f(t)$ living on a compact set $T$. In particular, we are interested in tail events associated with the integral $\int_Te^{f(t)}\,dt$. We construct a (non-Gaussian) random field whose distribution can be explicitly stated. This field approximates the conditional Gaussian random field $f$ (given that $\int_Te^{f(t)}\,dt$ exceeds a large value) in total variation. Based on this approximation, we show that the tail event of $\int_Te^{f(t)}\,dt$ is asymptotically equivalent to the tail event of $\sup_Tγ(t)$ where $γ(t)$ is …
We consider an affine process $X$ which is only observed up to an additive white noise, and we ask for its law, for some time $t > 0 $, conditional on all observations up to this time $ t $. This is a general, possibly high dimensional filtering problem which is not even locally approximately Gaussian, whence essentially only particle filtering methods remain as solution techniques. In this work we present an efficient numerical solution by introducing an approximate filter for which conditional characteristic functions can be calculated by solving a system of generalized Riccati differential …
This paper is the first part of a series of papers on filtering for partially observed jump diffusions satisfying a stochastic differential equation driven by Wiener processes and Poisson martingale measures. The coefficients of the equation only satisfy appropriate growth conditions. Some results in filtering theory of diffusion processes are extended to jump diffusions and equations for the time evolution of the conditional distribution and the unnormalised conditional distribution of the unobserved process at time $t$, given the observations until $t$, are presented.
This article is concerned with the exponential stability and the uniform propagation of chaos properties of a class of Extended Ensemble Kalman-Bucy filters with respect to the time horizon. This class of nonlinear filters can be interpreted as the conditional expectations of nonlinear McKean Vlasov type diffusions with respect to the observation process. In contrast with more conventional Langevin nonlinear drift type processes, the mean field interaction is encapsulated in the covariance matrix of the diffusion. The main results discussed in the article are quantitative estimates of the expo…
Filtering has had a profound impact as a device of perceiving information and deriving agent expectations in dynamic economic models. For an abstract economic system, this paper shows that the foundation of applying the filtering method corresponds to the existence of a conditional expectation as an equilibrium process. Agent-based rational behavior of looking backward and looking forward is generalized to a conditional expectation process where the economic system is approximated by a class of models, which can be represented and estimated without information loss. The proposed framework eluc…
Uniform convergence of conditional distributions for one-dimensional diffusion processes
School of Mathematics and Statistics, Hunan University of Technology and Business（湖南工业大学与商学院数学与统计学院）；School of Mathematics and Computational Science, Xiangtan University（湘潭大学数学与计算科学学院）
Transformers Can Solve Non-Linear and Non-Markovian Filtering Problems in Continuous Time For Conditionally Gaussian Signals
Department of Mathematics, University of Oxford（牛津大学数学系）；Oxford Man Institute for Quantitative Finance, University of Oxford（牛津大学牛津曼量化金融研究所）；Department of Mathematics, McMaster University（麦克马斯特大学数学系）；Vector Institute（向量研究所）
The filtering equations associated to a partially observed jump diffusion model $(Z_t)_{t\in [0,T]}=(X_t,Y_t)_{t\in [0,T]}$, driven by Wiener processes and Poisson martingale measures are considered. Building on results from two preceding articles on the filtering equations, the regularity of the conditional density of the signal $X_t$, given observations $(Y_s)_{s\in [0,t]}$, is investigated, when the conditional density of $X_0$ given $Y_0$ exists and belongs to a Sobolev space, and the coefficients satisfy appropriate smoothness and growth conditions.
In a Bayesian setting, inverse problems and uncertainty quantification (UQ) --- the propagation of uncertainty through a computational (forward) model --- are strongly connected. In the form of conditional expectation the Bayesian update becomes computationally attractive. We give a detailed account of this approach via conditional approximation, various approximations, and the construction of filters. Together with a functional or spectral approach for the forward UQ there is no need for time-consuming and slowly convergent Monte Carlo sampling. The developed sampling-free non-linear Bayesian…
Consider a pair of random vectors $(\mathbf{X},\mathbf{Y}) $ and the conditional expectation operator $\mathbb{E}[\mathbf{X}|\mathbf{Y}=\mathbf{y}]$. This work studies analytic properties of the conditional expectation by characterizing various derivative identities. The paper consists of two parts. In the first part of the paper, a general derivative identity for the conditional expectation is derived. Specifically, for the Markov chain $\mathbf{U} \leftrightarrow \mathbf{X} \leftrightarrow \mathbf{Y}$, a compact expression for the Jacobian matrix of $\mathbb{E}[\mathbf{U}|\mathbf{Y}=\mathbf{…
The paper studies the problem of filtering a discrete-time linear system observed by a network of sensors. The sensors share a common communication medium to the estimator and transmission is bit and power budgeted. Under the assumption of conditional Gaussianity of the signal process at the estimator (which may be ensured by observation packet acknowledgements), the conditional prediction error covariance of the optimum mean-squared error filter is shown to evolve according to a random dynamical system (RDS) on the space of non-negative definite matrices. Our RDS formalism does not depend on …
The problem of stability of the optimal filter is revisited. The optimal filter (or filtering process) is the conditional probability of the current state of some stochastic process (the signal process), given both present and past values of another process (the observation process). Typically the filtering process satisfies a dynamical equation, and the question investigated here concerns the stability of this dynamics. In contrast to previous work, signal processes given by the iterations of a deterministic mapping $f$ are considered, with only the initial condition being random. While the s…
It has been established under very general conditions that the ergodic properties of Markov processes are inherited by their conditional distributions given partial information. While the existing theory provides a rather complete picture of classical filtering models, many infinite-dimensional problems are outside its scope. Far from being a technical issue, the infinite-dimensional setting gives rise to surprising phenomena and new questions in filtering theory. The aim of this paper is to discuss some elementary examples, conjectures, and general theory that arise in this setting, and to hi…
Imperial College London（帝国理工学院）；University of Oslo（奥斯陆大学）
The filtering equations govern the evolution of the conditional distribution of a signal process given partial, and possibly noisy, observations arriving sequentially in time. Their numerical approximation plays a central role in many real-life applications, including numerical weather prediction, finance and engineering. One of the classical approaches to approximate the solution of the filtering equations is to use a PDE inspired method, called the splitting-up method, initiated by Gyongy, Krylov, LeGland, among other contributors. This method, and other PDE based approaches, have particular…
The relation between rate distortion function (RDF) and Bayesian filtering theory is discussed. The relation is established by imposing a causal or realizability constraint on the reconstruction conditional distribution of the RDF, leading to the definition of a causal RDF. Existence of the optimal reconstruction distribution of the causal RDF is shown using the topology of weak convergence of probability measures. The optimal non-stationary causal reproduction conditional distribution of the causal RDF is derived in closed form; it is given by a set of recursive equations which are computed b…
Combinations of neural ODEs with recurrent neural networks (RNN), like GRU-ODE-Bayes or ODE-RNN are well suited to model irregularly observed time series. While those models outperform existing discrete-time approaches, no theoretical guarantees for their predictive capabilities are available. Assuming that the irregularly-sampled time series data originates from a continuous stochastic process, the $L^2$-optimal online prediction is the conditional expectation given the currently available information. We introduce the Neural Jump ODE (NJ-ODE) that provides a data-driven approach to learn, co…
We consider the problem of high-dimensional filtering of state-space models (SSMs) at discrete times. This problem is particularly challenging as analytical solutions are typically not available and many numerical approximation methods can have a cost that scales exponentially with the dimension of the hidden state. Inspired by lag-approximation methods for the smoothing problem, we introduce a lagged approximation of the smoothing distribution that is necessarily biased. For certain classes of SSMs, particularly those that forget the initial condition exponentially fast in time, the bias of o…
