The aim of this paper is to compare three regularized particle filters in an online data processing context. We carry out the comparison in terms of hidden states filtering and parameters estimation, considering a Bayesian paradigm and a univariate stochastic volatility model. We discuss the use of an improper prior distribution in the initialization of the filtering procedure and show that the regularized Auxiliary Particle Filter (APF) outperforms the regularized Sequential Importance Sampling (SIS) and the regularized Sampling Importance Resampling (SIR).
We analyze the performance of different resampling strategies for the regularized particle filter regarding parameter estimation. We show in particular, building on analytical insight obtained in the linear Gaussian case, that resampling systematically can prevent the filtered density from converging towards the true posterior distribution. We discuss several means to overcome this limitation, including kernel bandwidth modulation, and provide evidence that the resulting particle filter clearly outperforms traditional bootstrap particle filters. Our results are supported by numerical simulatio…
Aalto University（阿尔托大学）；University of Oxford（牛津大学）
Particle Filtering (PF) methods are an established class of procedures for performing inference in non-linear state-space models. Resampling is a key ingredient of PF, necessary to obtain low variance likelihood and states estimates. However, traditional resampling methods result in PF-based loss functions being non-differentiable with respect to model and PF parameters. In a variational inference context, resampling also yields high variance gradient estimates of the PF-based evidence lower bound. By leveraging optimal transport ideas, we introduce a principled differentiable particle filter …
For particle filters and ensemble Kalman filters it is of practical importance to understand how and why data assimilation methods can be effective when used with a fixed small number of particles, since for many large-scale applications it is not practical to deploy algorithms close to the large particle limit asymptotic. In this paper we address this question for particle filters and, in particular, study their accuracy (in the small noise limit) and ergodicity (for noisy signal and observation) without appealing to the large particle number limit. We first overview the accuracy and minoriza…
In the modern age of social media and networks, graph representations of real-world phenomena have become an incredibly useful source to mine insights. Often, we are interested in understanding how entities in a graph are interconnected. The Graph Neural Network (GNN) has proven to be a very useful tool in a variety of graph learning tasks including node classification, link prediction, and edge classification. However, in most of these tasks, the graph data we are working with may be noisy and may contain spurious edges. That is, there is a lot of uncertainty associated with the underlying gr…
Geometry-Consistent Bayesian Filtering under Structural Model Uncertainty: A Geometric Projection Particle Filter
U R Rao Satellite Centre, Indian Space Research Organisation (ISRO)（印度空间研究组织乌拉·拉奥卫星中心）；Centre for Computational and Data Sciences, Indian Institute of Science（印度科学学院计算与数据科学中心）
Advances in digital sensors, digital data storage and communications have resulted in systems being capable of accumulating large collections of data. In the light of dealing with the challenges that massive data present, this work proposes solutions to inference and filtering problems within the Bayesian framework. Two novel Bayesian inference algorithms are developed for non-linear and non-Gaussian state space models, able to deal with large volumes of data (or observations). These are sequential Markov chain Monte Carlo (MCMC) approaches relying on two key ideas: 1) subsample the massive da…
A methodology to perform topological regularization via information filtering network is introduced. This methodology can be directly applied to covariance selection problem providing an instrument for sparse probabilistic modeling with both linear and non-linear multivariate probability distributions such as the elliptical and generalized hyperbolic families. It can also be directly implemented for $L_0$-norm regularized multicollinear regression. In this paper, I describe in detail an application to sparse modeling with multivariate Student-t. A specific $L_0$-norm regularized expectation-ma…
We address the problem of constructing varying-coefficient models based on basis expansions along with the technique of regularization. A crucial point in our modeling procedure is the selection of smoothing parameters in the regularization method. In order to choose the parameters objectively, we derive model selection criteria from the viewpoints of information-theoretic and Bayesian approach. We demonstrate the effectiveness of proposed modeling strategy through Monte Carlo simulations and analyzing a real data set.
Regularization is a common tool in variational inverse problems to impose assumptions on the parameters of the problem. One such assumption is sparsity, which is commonly promoted using lasso and total variation-like regularization. Although the solutions to many such regularized inverse problems can be considered as points of maximum probability of well-chosen posterior distributions, samples from these distributions are generally not sparse. In this paper, we present a framework for implicitly defining a probability distribution that combines the effects of sparsity imposing regularization w…
This article discusses a partially adapted particle filter for estimating the likelihood of a nonlinear structural econometric state space models whose state transition density cannot be expressed in closed form. The filter generates the disturbances in the state transition equation and allows for multiple modes in the conditional disturbance distribution. The particle filter produces an unbiased estimate of the likelihood and so can be used to carry out Bayesian inference in a particle Markov chain Monte Carlo framework. We show empirically that when the signal to noise ratio is high, the new…
Bayesian filtering aims at tracking sequentially a hidden process from an observed one. In particular, sequential Monte Carlo (SMC) techniques propagate in time weighted trajectories which represent the posterior probability density function (pdf) of the hidden process given the available observations. On the other hand, Conditional Monte Carlo (CMC) is a variance reduction technique which replaces the estimator of a moment of interest by its conditional expectation given another variable. In this paper we show that up to some adaptations, one can make use of the time recursive nature of SMC a…
Proximal Markov Chain Monte Carlo is a novel construct that lies at the intersection of Bayesian computation and convex optimization, which helped popularize the use of nondifferentiable priors in Bayesian statistics. Existing formulations of proximal MCMC, however, require hyperparameters and regularization parameters to be prespecified. In this work, we extend the paradigm of proximal MCMC through introducing a novel new class of nondifferentiable priors called epigraph priors. As a proof of concept, we place trend filtering, which was originally a nonparametric regression problem, in a para…
In this manuscript, a general method for deriving filtering algorithms that involve a network of interconnected Bayesian filters is proposed. This method is based on the idea that the processing accomplished inside each of the Bayesian filters and the interactions between them can be represented as message passing algorithms over a proper graphical model. The usefulness of our method is exemplified by developing new filtering techniques, based on the interconnection of a particle filter and an extended Kalman filter, for conditionally linear Gaussian systems. Numerical results for two specific…
Marginalization techniques are presented for the Bayesian filtering problem under the assumption of Gaussian priors and posteriors and a set of sequentially more constraining state space model assumptions. The techniques provide the capabilities to 1) reduce the filtering problem to essential marginal moment integrals, 2) combine model and numerical approximations to the moment integrals, and 3) decouple modelling and system composition. Closed-form expressions of the posterior means and covariances are developed as functions of subspace projection matrices, subsystem models, and the marginal …
Recently, a novel method for developing filtering algorithms, based on the interconnection of two Bayesian filters and called double Bayesian filtering, has been proposed. In this manuscript we show that the same conceptual approach can be exploited to devise a new smoothing method, called double Bayesian smoothing. A double Bayesian smoother combines a double Bayesian filter, employed in its forward pass, with the interconnection of two backward information filters used in its backward pass. As a specific application of our general method, a detailed derivation of double Bayesian smoothing al…
Markov chain Monte Carlo (MCMC) methods are powerful computational tools for analysis of complex statistical problems. However, their computational efficiency is highly dependent on the chosen proposal distribution, which is generally difficult to find. One way to solve this problem is to use adaptive MCMC algorithms which automatically tune the statistics of a proposal distribution during the MCMC run. A new adaptive MCMC algorithm, called the variational Bayesian adaptive Metropolis (VBAM) algorithm, is developed. The VBAM algorithm updates the proposal covariance matrix using the variationa…
Regularized infill criteria for multi-objective Bayesian optimization with application to aircraft design
ISAE-SUPAERO（法国高等航空航天学院）；Université de Toulouse（图卢兹大学）；MINES ALBI（阿尔比矿业学院）；UPS（图卢兹第三大学）；INSA（图卢兹国立应用科学学院）；CNRS（法国国家科学研究中心）；ONERA（法国航空航天研究院）；DLR（德国航空航天中心）
