Given a linear dynamical system affected by stochastic noise, we consider the problem of selecting an optimal set of sensors (at design-time) to minimize the trace of the steady state a priori or a posteriori error covariance of the Kalman filter, subject to certain selection budget constraints. We show the fundamental result that there is no polynomial-time constant-factor approximation algorithm for this problem. This contrasts with other classes of sensor selection problems studied in the literature, which typically pursue constant-factor approximations by leveraging greedy algorithms and submodularity (or supermodularity) of the cost function. Here, we provide a specific example showing that greedy algorithms can perform arbitrarily poorly for the problem of design-time sensor selection for Kalman filtering. We then study the problem of attacking (i.e., removing) a set of installed sensors, under predefined attack budget constraints, to maximize the trace of the steady state a priori or a posteriori error covariance of the Kalman filter. Again, we show that there is no polynomial-time constant-factor approximation algorithm for this problem, and show specifically that greedy algorithms can perform arbitrarily poorly.
This paper investigates the distributed Kalman filtering (DKF) from distributed optimization viewpoint. Motivated by the fact that Kalman filtering is a maximum a posteriori estimation (MAP) problem, which is a quadratic optimization problem, we reformulate DKF problem as a consensus optimization problem, resulting in that it can be solved by many existing distributed optimization algorithms. A new DKF algorithm employing the dual ascent method is proposed, and its stability is proved under mild assumptions. The performance of the proposed algorithm is evaluated through numerical experiments.
We consider the problem of randomly choosing the sensors of a linear time-invariant dynamical system subject to process and measurement noise. We sample the sensors independently and from the same distribution. We measure the performance of a Kalman filter by its estimation error covariance. Using tools from random matrix theory, we derive probabilistic bounds on the estimation error covariance in the semi-definite sense. We indirectly improve the performance of our Kalman filter for the maximum eigenvalue metric and show that under certain conditions the optimal sampling distribution that min…
This paper investigates the use of extended Kalman filtering to train recurrent neural networks with rather general convex loss functions and regularization terms on the network parameters, including $\ell_1$-regularization. We show that the learning method is competitive with respect to stochastic gradient descent in a nonlinear system identification benchmark and in training a linear system with binary outputs. We also explore the use of the algorithm in data-driven nonlinear model predictive control and its relation with disturbance models for offset-free closed-loop tracking.
We consider the problem of distributed Kalman filtering for sensor networks in the case there is a limit in data transmission and there is model uncertainty. More precisely, we propose a distributed filtering strategy with event-triggered communication in which the state estimators are computed according to the least favorable model. The latter belongs to a ball (in Kullback-Leibler topology) about the nominal model. We also present a preliminary numerical example in order to test the performance of the proposed strategy.
A generalized Kalman-Bucy model under model uncertainty and a corresponding robust problem are studied in this paper. We find that this robust problem is equivalent to an estimate problem under a sublinear operator. By Girsanov transformation and the minimax theorem, we prove that this problem can be reformulated as a classical Kalman-Bucy filtering problem under a new probability measure. The equation which governs the optimal estimator is obtained. Moreover, the optimal estimator can be decomposed into the classical optimal estimator and a term related to model uncertainty.
We consider the problem of distributed Kalman filtering for sensor networks in the case there are constraints in data transmission and there is model uncertainty. More precisely, we propose two distributed filtering strategies with event-triggered communication where the state estimators are computed according to the least favorable model. The latter belongs to a ball about the nominal model. We also show that both the methods are stable in the sense that the mean-square of the state estimation error is bounded in all the nodes.
We consider a robust filtering problem where the nominal state space model is not reachable and different from the actual one. We propose a robust Kalman filter which solves a dynamic game: one player selects the least-favorable model in a given ambiguity set, while the other player designs the optimum filter for the least-favorable model. It turns out that the robust filter is governed by a low-rank risk sensitive-like Riccati equation. Finally, simulation results show the effectiveness of the proposed filter.
We study the problem of distributed Kalman filtering for sensor networks in the presence of model uncertainty. More precisely, we assume that the actual state-space model belongs to a ball, in the Kullback-Leibler topology, about the nominal state-space model and whose radius reflects the mismatch modeling budget allowed for each time step. We propose a distributed Kalman filter with diffusion step which is robust with respect to the aforementioned model uncertainty. Moreover, we derive the corresponding least favorable performance. Finally, we check the effectiveness of the proposed algorithm…
We consider a robust filtering problem where the robust filter is designed according to the least favorable model belonging to a ball about the nominal model. In this approach, the ball radius specifies the modeling error tolerance and the least favorable model is computed by performing a Riccati-like backward recursion. We show that this recursion converges provided that the tolerance is sufficiently small.
The universality of the celebrated Kalman filtering can be found in control theory. The Kalman filter has found its striking applications in sophisticated autonomous systems and smart products, which are attributed to its realization in a single complex chip. In this paper, we revisit the Kalman filter from the perspective of conditional characteristic function evolution and Ito calculus and develop three Kalman filtering Theorems and their formal proof. Most notably, this paper reveals the following: (i) Kalman filtering equations are a consequence of the evolution of conditional characterist…
We consider a robust state space filtering problem in the case that the transition probability density is unknown and possibly degenerate. The resulting robust filter has a Kalman-like structure and solves a minimax game: the nature selects the least favorable model in a prescribed ambiguity set which also contains non-Gaussian probability densities, while the other player designs the optimum filter for the least favorable model. It turns out that the resulting robust filter is characterized by a Riccati-like iteration evolving on the cone of the positive semidefinite matrices. Moreover, we st…
In this paper, we study a generalized Kalman-Bucy filtering problem under uncertainty. The drift uncertainty for both signal process and observation process is considered and the attitude to uncertainty is characterized by a convex operator (convex risk measure). The optimal filter or the minimum mean square estimator (MMSE) is calculated by solving the minimum mean square estimation problem under a convex operator. In the first part of this paper, this estimation problem is studied under g-expectation which is a special convex operator. For this case, we prove that there exists a worst-case p…
In this article, we consider the implications of unobservable subspaces in the construction of a Kalman filter. In particular, we consider dynamical systems which are invariant with respect to a group action, and which are therefore unobservable in the group direction. We obtain reduced propagation and measurement equations that are invariant with respect to the group action, and we decompose the state space into unobservable and observable parts. Based on the decomposition, we propose a reduced Bayesian inference method, which exhibits superior accuracy for orientation and position estimation…
In this paper, we consider the task of designing a Kalman Filter (KF) for an unknown and partially observed autonomous linear time invariant system driven by process and sensor noise. To do so, we propose studying the following two step process: first, using system identification tools rooted in subspace methods, we obtain coarse finite-data estimates of the state-space parameters and Kalman gain describing the autonomous system; and second, we use these approximate parameters to design a filter which produces estimates of the system state. We show that when the system identification step prod…
We consider the problem of selecting an optimal set of sensor precisions to estimate the states of a non-linear dynamical system using an Ensemble Kalman filter and an Unscented Kalman filter, which uses random and deterministic ensembles respectively. Specifically, the goal is to choose at run-time, a sparse set of sensor precisions for active-sensing that satisfies certain constraints on the estimated state covariance. In this paper, we show that this sensor precision selection problem is a semidefinite programming problem when we use l1 norm over precision vector as the surrogate measure to…
École Polytechnique Fédérale de Lausanne（洛桑联邦理工学院）；TU Delft（代尔夫特理工大学）
We study a distributionally robust mean square error estimation problem over a nonconvex Wasserstein ambiguity set containing only normal distributions. We show that the optimal estimator and the least favorable distribution form a Nash equilibrium. Despite the non-convex nature of the ambiguity set, we prove that the estimation problem is equivalent to a tractable convex program. We further devise a Frank-Wolfe algorithm for this convex program whose direction-searching subproblem can be solved in a quasi-closed form. Using these ingredients, we introduce a distributionally robust Kalman filt…
In this paper, we present an optimization-based formulation for privacy-utility tradeoff in the Ensemble and Unscented Kalman filtering framework, with a focus on space situational awareness. Privacy and utility are defined in terms of a lower and an upper bound on the state estimation error covariance, respectively. Synthetic sensor noise is used to satisfy these bounds and is determined by solving an optimization problem. Given privacy and utility bounds, we present optimization problem formulations to determine a) the maximum noise for which utility is satisfied or the estimation errors are…
Virginia Tech（弗吉尼亚理工大学）；Computational Science Laboratory（计算科学实验室）
Data assimilation is a Bayesian inference process that obtains an enhanced understanding of a physical system of interest by fusing information from an inexact physics-based model, and from noisy sparse observations of reality. The multifidelity ensemble Kalman filter (MFEnKF) recently developed by the authors combines a full-order physical model and a hierarchy of reduced order surrogate models in order to increase the computational efficiency of data assimilation. The standard MFEnKF uses linear couplings between models, and is statistically optimal in case of Gaussian probability densities.…
It is often said that control and estimation problems are in duality. Recently, in (Aubin-Frankowski,2021), we found new reproducing kernels in Linear-Quadratic optimal control by focusing on the Hilbert space of controlled trajectories, allowing for a convenient handling of state constraints and meeting points. We now extend this viewpoint to estimation problems where it is known that kernels are the covariances of stochastic processes. Here, the Markovian Gaussian processes stem from the linear stochastic differential equations describing the continuous-time dynamics and observations. Taking…
We introduce Kalman Gradient Descent, a stochastic optimization algorithm that uses Kalman filtering to adaptively reduce gradient variance in stochastic gradient descent by filtering the gradient estimates. We present both a theoretical analysis of convergence in a non-convex setting and experimental results which demonstrate improved performance on a variety of machine learning areas including neural networks and black box variational inference. We also present a distributed version of our algorithm that enables large-dimensional optimization, and we extend our algorithm to SGD with momentum…
