We prove that for linear, discrete, time-varying, deterministic system (perfect model) with noisy outputs, the Riccati transformation in the Kalman filter asymptotically bounds the rank of the forecast and the analysis error covariance matrices to be less than or equal to the number of non-negative Lyapunov exponents of the system. Further, the support of these error covariance matrices is shown to be confined to the space spanned by the unstable-neutral backward Lyapunov vectors, providing the theoretical justification for the methodology of the algorithms that perform assimilation only in the unstable-neutral subspace. The equivalent property of the autonomous system is investigated as a special case.
One of the most interesting questions in control theory is that of constructing observers. Observers compute estimates of the internal states of a dynamical system, using data provided by measurement probes or partial state information. For linear systems, Luenberger observers (also known as "deterministic Kalman filters" since they amount to Kalman filters designed without regard to the statistics of measurement noise) provide a general solution, but, for nonlinear systems, establishing generally applicable conditions for existence and convergence of observers is an open and active area of re…
Several variations of the Kalman filter algorithm, such as the extended Kalman filter (EKF) and the unscented Kalman filter (UKF), are widely used in science and engineering applications. In this paper, we introduce two algorithms of sparsity-based Kalman filters, namely the sparse UKF and the progressive EKF. The filters are designed specifically for problems with very high dimensions. Different from various types of ensemble Kalman filters (EnKFs) in which the error covariance is approximated using a set of dense ensemble vectors, the algorithms developed in this paper are based on sparse ma…
In this paper an unscented Kalman filter with guaranteed positive semidefinite state covariance is proposed by calculating the nearest symmetric positive definite matrix in Frobenius norm and is applied to power system dynamic state estimation. The proposed method is tested on NPCC 48-machine 140-bus system and the results validate its effectiveness.
The characteristics of the model dynamics are critical in the performance of (ensemble) Kalman filters. In particular, as emphasized in the seminal work of Anna Trevisan and co-authors, the error covariance matrix is asymptotically supported by the unstable-neutral subspace only, i.e., it is spanned by the backward Lyapunov vectors with non-negative exponents. This behavior is at the core of algorithms known as Assimilation in the Unstable Subspace, although a formal proof was still missing. This paper provides the analytical proof of the convergence of the Kalman filter covariance matrix onto…
The optimal gain matrix of the Kalman filter is often derived by minimizing the trace of the posterior covariance matrix. Here, I show that the Kalman gain also minimizes the determinant of the covariance matrix, a quantity known as the generalized variance. When the error distributions are Gaussian, the differential entropy is also minimized.
This paper presents a computationally fast algorithm for estimating, both, the system and observation noise covariances of nonlinear dynamics, that can be used in an ensemble Kalman filtering framework. The new method is a modification of Belanger's recursive method, to avoid an expensive computational cost in inverting error covariance matrices of product of innovation processes of different lags when the number of observations becomes large. When we use only product of innovation processes up to one-lag, the computational cost is indeed comparable to a recently proposed method by Berry-Sauer…
A priori state vector and error covariance computation for the Unscented Kalman Filter (UKF) is described. The original UKF propagates multiple sigma points to compute the a priori mean state vector and the error covariance, resulting in a higher computational time compared to the Extended Kalman Filter (EKF). In the proposed method, the posterior mean state vector is propagated and then the sigma points at the current time step are calculated using the first-order Taylor Series approximation. This reduces the computation time significantly, as demonstrated using two example applications which…
Kalman filtering and smoothing algorithms are used in many areas, including tracking and navigation, medical applications, and financial trend filtering. One of the basic assumptions required to apply the Kalman smoothing framework is that error covariance matrices are known and given. In this paper, we study a general class of inference problems where covariance matrices can depend functionally on unknown parameters. In the Kalman framework, this allows modeling situations where covariance matrices may depend functionally on the state sequence being estimated. We present an extended formulati…
This note examines the influence of covariance inflation on the distance between the measured observation and the simulated (or predicted) observation with respect to the state estimate. In order for the aforementioned distance to be bounded in a certain interval, some sufficient conditions are derived, indicating that the covariance inflation factor should be bounded in a certain interval, and that the inflation bounds are related to the maximum and minimum eigenvalues of certain matrices. Implications of these analytic results are discussed, and a numerical experiment is presented to verify …
The Kalman filter (KF) and the extended Kalman filter (EKF) are well established techniques for state estimation. However, the choice of the filter tuning parameters still poses a major challenge for the engineers [1]. In the present work, two new costs have been proposed for determining the filter tuning parameters on the basis of the innovation covariance. This provides a cost function based method for the selection of suitable combination(s) of filter tuning parameters in order to ensure the design of a KF or an EKF having an optimally balanced RMSE performance. Index Terms-Kalman filter, t…
We consider the problem of selecting an optimal set of sensor precisions to estimate the states of a non-linear dynamical system using an Ensemble Kalman filter and an Unscented Kalman filter, which uses random and deterministic ensembles respectively. Specifically, the goal is to choose at run-time, a sparse set of sensor precisions for active-sensing that satisfies certain constraints on the estimated state covariance. In this paper, we show that this sensor precision selection problem is a semidefinite programming problem when we use l1 norm over precision vector as the surrogate measure to…
Ensemble Kalman Inversion as an Inertial Interacting Particle System
Institute for Geometry and Applied Mathematics, RWTH Aachen University（亚琛工业大学几何与应用数学研究所）；Department of Mathematics and Applied Mathematics, University of Pretoria（比勒陀利亚大学数学与应用数学系）；Department of Mathematics, Sapienza University of Rome（罗马第一大学数学系）
This paper is concerned with optimality and stability analysis of a family of ensemble Kalman filter (EnKF) algorithms. EnKF is commonly used as an alternative to the Kalman filter for high-dimensional problems, where storing the covariance matrix is computationally expensive. The algorithm consists of an ensemble of interacting particles driven by a feedback control law. The control law is designed such that, in the linear Gaussian setting and asymptotic limit of infinitely many particles, the mean and covariance of the particles follow the exact mean and covariance of the Kalman filter. The …
We consider the problem of randomly choosing the sensors of a linear time-invariant dynamical system subject to process and measurement noise. We sample the sensors independently and from the same distribution. We measure the performance of a Kalman filter by its estimation error covariance. Using tools from random matrix theory, we derive probabilistic bounds on the estimation error covariance in the semi-definite sense. We indirectly improve the performance of our Kalman filter for the maximum eigenvalue metric and show that under certain conditions the optimal sampling distribution that min…
The ensemble Kalman inversion (EKI) is a particle based method which has been introduced as the application of the ensemble Kalman filter to inverse problems. In practice it has been widely used as derivative-free optimization method in order to estimate unknown parameters from noisy measurement data. For linear forward models the EKI can be viewed as gradient flow preconditioned by a certain sample covariance matrix. Through the preconditioning the resulting scheme remains in a finite dimensional subspace of the original high-dimensional (or even infinite dimensional) parameter space and can …
Given a linear dynamical system affected by noise, we study the problem of optimally placing sensors (at design-time) subject to a sensor placement budget constraint in order to minimize the trace of the steady-state error covariance of the corresponding Kalman filter. While this problem is NP-hard in general, we consider the underlying graph associated with the system dynamics matrix, and focus on the case when there is a single input at one of the nodes in the graph. We provide an optimal strategy (computed in polynomial-time) to place the sensors over the network. Next, we consider the prob…
We derive a reduced-order state estimator for discrete-time infinite dimensional linear systems with finite dimensional Gaussian input and output noise. This state estimator is the optimal one-step estimate that takes values in a fixed finite dimensional subspace of the system's state space --- consider, for example, a Finite Element space. We then derive a Riccati difference equation for the error covariance and use sensitivity analysis to obtain a bound for the error of the state estimate due to the state space discretization.
The paper presents the gossip interactive Kalman filter (GIKF) for distributed Kalman filtering for networked systems and sensor networks, where inter-sensor communication and observations occur at the same time-scale. The communication among sensors is random; each sensor occasionally exchanges its filtering state information with a neighbor depending on the availability of the appropriate network link. We show that under a weak distributed detectability condition: 1. the GIKF error process remains stochastically bounded, irrespective of the instability properties of the random process dynami…
We present a method of using classical wavelet based multiresolution analysis to separate scales in model and observations during data assimilation with the ensemble Kalman filter. In many applications, the underlying physics of a phenomena involve the interaction of features at multiple scales. Blending of observational and model error across scales can result in large forecast inaccuracies since large errors at one scale are interpreted as inexact data at all scales. Our method uses a transformation of the observation operator in order to separate the information from different scales of the…
In this paper we are concerned with the error-covariance lower-bounding problem in Kalman filtering: a sensor releases a set of measurements to the data fusion/estimation center, which has a perfect knowledge of the dynamic model, to allow it to estimate the states, while preventing it to estimate the states beyond a given accuracy. We propose a measurement noise manipulation scheme to ensure lower-bound on the estimation accuracy of states. Our proposed method ensures lower-bound on the steady state estimation error of Kalman filter, using mathematical tools from eigen value analysis.
It is often said that control and estimation problems are in duality. Recently, in (Aubin-Frankowski,2021), we found new reproducing kernels in Linear-Quadratic optimal control by focusing on the Hilbert space of controlled trajectories, allowing for a convenient handling of state constraints and meeting points. We now extend this viewpoint to estimation problems where it is known that kernels are the covariances of stochastic processes. Here, the Markovian Gaussian processes stem from the linear stochastic differential equations describing the continuous-time dynamics and observations. Taking…
Given a linear dynamical system affected by stochastic noise, we consider the problem of selecting an optimal set of sensors (at design-time) to minimize the trace of the steady state a priori or a posteriori error covariance of the Kalman filter, subject to certain selection budget constraints. We show the fundamental result that there is no polynomial-time constant-factor approximation algorithm for this problem. This contrasts with other classes of sensor selection problems studied in the literature, which typically pursue constant-factor approximations by leveraging greedy algorithms and s…
Duality of control and estimation allows mapping recent advances in data-guided control to the estimation setup. This paper formalizes and utilizes such a mapping to consider learning the optimal (steady-state) Kalman gain when process and measurement noise statistics are unknown. Specifically, building on the duality between synthesizing optimal control and estimation gains, the filter design problem is formalized as direct policy learning. In this direction, the duality is used to extend existing theoretical guarantees of direct policy updates for Linear Quadratic Regulator (LQR) to establis…
The unscented Kalman inversion (UKI) presented in [1] is a general derivative-free approach to solving the inverse problem. UKI is particularly suitable for inverse problems where the forward model is given as a black box and may not be differentiable. The regularization strategy and convergence property of the UKI are thoroughly studied, and the method is demonstrated effectively handling noisy observation data and solving chaotic inverse problems. In this paper, we aim to make the UKI more efficient in terms of computational and memory costs for large scale inverse problems. We take advantag…
