Nature has evolved many molecular machines such as kinesin, myosin, and the rotary flagellar motor powered by an ion current from the mitochondria. Direct observation of the step-like motion of these machines with time series from novel experimental assays has recently become possible. These time series are corrupted by molecular and experimental noise that requires removal, but classical signal processing is of limited use for recovering such step-like dynamics. This paper reports simple, novel Bayesian filters that are robust to step-like dynamics in noise, and introduce an L1-regularized, global filter whose sparse solution can be rapidly obtained by standard convex optimization methods. We show these techniques outperforming classical filters on simulated time series in terms of their ability to accurately recover the underlying step dynamics. To show the techniques in action, we extract step-like speed transitions from Rhodobacter sphaeroides flagellar motor time series. Code implementing these algorithms available from http://www.eng.ox.ac.uk/samp/members/max/software/.
The level-set method is a prominent approach to modelling the evolution of a fire over time based on a characterised rate of spread. It however does not provide a direct means for assimilating new data and quantifying uncertainty. Fire front predictions can be more accurate and agile if the models are able to assimilate data in real time. Furthermore, uncertainty estimation of the location and spread of the fire is critical for decision making. Using Bayesian filtering approaches, we extend the level-set method to allow for data assimilation and uncertainty quantification. We demonstrate these…
This paper presents a new modeling strategy for joint unsupervised analysis of multiple high-throughput biological studies. As in Multi-study Factor Analysis, our goals are to identify both common factors shared across studies and study-specific factors. Our approach is motivated by the growing body of high-throughput studies in biomedical research, as exemplified by the comprehensive set of expression data on breast tumors considered in our case study. To handle high-dimensional studies, we extend Multi-study Factor Analysis using a Bayesian approach that imposes sparsity. Specifically, we ge…
In this manuscript, a general method for deriving filtering algorithms that involve a network of interconnected Bayesian filters is proposed. This method is based on the idea that the processing accomplished inside each of the Bayesian filters and the interactions between them can be represented as message passing algorithms over a proper graphical model. The usefulness of our method is exemplified by developing new filtering techniques, based on the interconnection of a particle filter and an extended Kalman filter, for conditionally linear Gaussian systems. Numerical results for two specific…
In this manuscript a method for developing novel filtering algorithms through the parallel concatenation of two Bayesian filters is illustrated. Our description of this method, called turbo filtering, is based on a new graphical model; this allows us to efficiently describe both the processing accomplished inside each of the constituent filter and the interactions between them. This model is exploited to develop two new filtering algorithms for conditionally linear Gaussian systems. Numerical results for a specific dynamic system evidence that such filters can achieve a better complexity-accur…
Optimal Bayesian feature filtering (OBF) is a supervised screening method designed for biomarker discovery. In this article, we prove two major theoretical properties of OBF. First, optimal Bayesian feature selection under a general family of Bayesian models reduces to filtering if and only if the underlying Bayesian model assumes all features are mutually independent. Therefore, OBF is optimal if and only if one assumes all features are mutually independent, and OBF is the only filter method that is optimal under at least one model in the general Bayesian framework. Second, OBF under independ…
We formulate probabilistic numerical approximations to solutions of ordinary differential equations (ODEs) as problems in Gaussian process (GP) regression with non-linear measurement functions. This is achieved by defining the measurement sequence to consist of the observations of the difference between the derivative of the GP and the vector field evaluated at the GP---which are all identically zero at the solution of the ODE. When the GP has a state-space representation, the problem can be reduced to a non-linear Bayesian filtering problem and all widely-used approximations to the Bayesian f…
We propose a Bayesian nonparametric method for low-pass filtering that can naturally handle unevenly-sampled and noise-corrupted observations. The proposed model is constructed as a latent-factor model for time series, where the latent factors are Gaussian processes with non-overlapping spectra. With this construction, the low-pass version of the time series can be identified as the low-frequency latent component, and therefore it can be found by means of Bayesian inference. We show that the model admits exact training and can be implemented with minimal numerical approximations. Finally, the …
Sparse Gaussian Processes are a key component of high-throughput Bayesian Optimisation (BO) loops; however, we show that existing methods for allocating their inducing points severely hamper optimisation performance. By exploiting the quality-diversity decomposition of Determinantal Point Processes, we propose the first inducing point allocation strategy designed specifically for use in BO. Unlike existing methods which seek only to reduce global uncertainty in the objective function, our approach provides the local high-fidelity modelling of promising regions required for precise optimisation…
University of South Carolina（南卡罗来纳大学）；University of North Carolina at Charlotte（北卡罗来纳大学夏洛特分校）
Real-time nonlinear Bayesian filtering algorithms are overwhelmed by data volume, velocity and increasing complexity of computational models. In this paper, we propose a novel ensemble based nonlinear Bayesian filtering approach which only requires a small number of simulations and can be applied to high-dimensional systems in the presence of intractable likelihood functions. The proposed approach uses linear latent projections to estimate the joint probability distribution between states, parameters, and observables using a mixture of Gaussian components generated by the reconstruction error …
We formulate the problem of neural network optimization as Bayesian filtering, where the observations are the backpropagated gradients. While neural network optimization has previously been studied using natural gradient methods which are closely related to Bayesian inference, they were unable to recover standard optimizers such as Adam and RMSprop with a root-mean-square gradient normalizer, instead getting a mean-square normalizer. To recover the root-mean-square normalizer, we find it necessary to account for the temporal dynamics of all the other parameters as they are geing optimized. The…
University of Warwick（华威大学）；The Alan Turing Institute（艾伦·图灵研究所）
We introduce a framework for inference in general state-space hidden Markov models (HMMs) under likelihood misspecification. In particular, we leverage the loss-theoretic perspective of Generalized Bayesian Inference (GBI) to define generalised filtering recursions in HMMs, that can tackle the problem of inference under model misspecification. In doing so, we arrive at principled procedures for robust inference against observation contamination by utilising the $β$-divergence. Operationalising the proposed framework is made possible via sequential Monte Carlo methods (SMC), where most standard…
Bayesian computation for filtering and forecasting analysis is developed for a broad class of dynamic models. The ability to scale-up such analyses in non-Gaussian, nonlinear multivariate time series models is advanced through the introduction of a novel copula construction in sequential filtering of coupled sets of dynamic generalized linear models. The new copula approach is integrated into recently introduced multiscale models in which univariate time series are coupled via nonlinear forms involving dynamic latent factors representing cross-series relationships. The resulting methodology of…
Secondmind（Secondmind公司）；University of Cambridge（剑桥大学）
Sparse Gaussian Processes are a key component of high-throughput Bayesian optimisation (BO) loops -- an increasingly common setting where evaluation budgets are large and highly parallelised. By using representative subsets of the available data to build approximate posteriors, sparse models dramatically reduce the computational costs of surrogate modelling by relying on a small set of pseudo-observations, the so-called inducing points, in lieu of the full data set. However, current approaches to design inducing points are not appropriate within BO loops as they seek to reduce global uncertain…
School of Vehicle and Mobility, Tsinghua University（清华大学车辆与出行学院）；Department of Advanced Manufacturing and Robotics, Peking University（北京大学先进制造与机器人系）；Department of Mathematical Sciences, Tsinghua University（清华大学数学科学系）；Beijing Institute of Mathematical Sciences and Applications（北京数学科学与应用研究院）
We introduce optimal transport based resampling in adaptive SMC. We consider elliptic inverse problems of inferring hydraulic conductivity from pressure measurements. We consider two parametrizations of hydraulic conductivity: by Gaussian random field, and by a set of scalar (non-)Gaussian distributed parameters and Gaussian random fields. We show that for scalar parameters optimal transport based SMC performs comparably to monomial based SMC but for Gaussian high-dimensional random fields optimal transport based SMC outperforms monomial based SMC. When comparing to ensemble Kalman inversion w…
This paper is concerned with the online estimation of a nonlinear dynamic system from a series of noisy measurements. The focus is on cases wherein outliers are present in-between normal noises. We assume that the outliers follow an unknown generating mechanism which deviates from that of normal noises, and then model the outliers using a Bayesian nonparametric model called Dirichlet process mixture (DPM). A sequential particle-based algorithm is derived for posterior inference for the outlier model as well as the state of the system to be estimated. The resulting algorithm is termed DPM based…
Coupling-Informed Transport Maps for Bayesian Filtering in Nonlinear Dynamical Systems
School of Mathematical Sciences, Tongji University（同济大学数学科学学院）；Key Laboratory of Intelligent Computing and Applications (Ministry of Education), Tongji University（智能计算与应用重点实验室（教育部））
Given a stationary state-space model that relates a sequence of hidden states and corresponding measurements or observations, Bayesian filtering provides a principled statistical framework for inferring the posterior distribution of the current state given all measurements up to the present time. For example, the Apollo lunar module implemented a Kalman filter to infer its location from a sequence of earth-based radar measurements and land safely on the moon. To perform Bayesian filtering, we require a measurement model that describes the conditional distribution of each observation given stat…
Regularity-informed data assimilation: A hierarchical Bayesian approach to ensemble Kalman filtering for hyperbolic conservation laws
Division of Applied Mathematics, Linköping University（林雪平大学应用数学系）；Center for Computational Science and Engineering, Massachusetts Institute of Technology（麻省理工学院计算科学与工程中心）
It has long been posited that there is a connection between the dynamical equations describing evolutionary processes in biology and sequential Bayesian learning methods. This manuscript describes new research in which this precise connection is rigorously established in the continuous time setting. Here we focus on a partial differential equation known as the Kushner-Stratonovich equation describing the evolution of the posterior density in time. Of particular importance is a piecewise smooth approximation of the observation path from which the discrete time filtering equations, which are sho…
In this paper, we address the problem of convergence of sequential variational inference filter (VIF) through the application of a robust variational objective and Hinf-norm based correction for a linear Gaussian system. As the dimension of state or parameter space grows, performing the full Kalman update with the dense covariance matrix for a large scale system requires increased storage and computational complexity, making it impractical. The VIF approach, based on mean-field Gaussian variational inference, reduces this burden through the variational approximation to the covariance usually i…
Filtering and smoothing with a generalised representation of uncertainty is considered. Here, uncertainty is represented using a class of outer measures. It is shown how this representation of uncertainty can be propagated using outer-measure-type versions of Markov kernels and generalised Bayesian-like update equations. This leads to a system of generalised smoothing and filtering equations where integrals are replaced by supremums and probability density functions are replaced by positive functions with supremum equal to one. Interestingly, these equations retain most of the structure found …
The effective reproduction number $R_t$ measures an infectious disease's transmissibility as the number of secondary infections in one reproduction time in a population having both susceptible and non-susceptible hosts. Current approaches do not quantify the uncertainty correctly in estimating $R_t$, as expected by the observed variability in contagion patterns. We elaborate on the Bayesian estimation of $R_t$ by improving on the Poisson sampling model of Cori et al. (2013). By adding an autoregressive latent process, we build a Dynamic Linear Model on the log of observed $R_t$s, resulting in …
