A general shape identification inverse problem is studied in a Bayesian framework. This problem requires the determination of the unknown shape of a domain in the Euclidean space from finite-dimensional observation data with some Gaussian random noise. Then, the stability of posterior is studied for observation data. For each point of the space, the conditional probability that the point is included in the unknown domain given the observation data is considered. The stability is also studied for this probability distribution. As a model problem for our inverse problem, a heat inverse problem is considered. This problem requires the determination of the unknown shape of cavities in a heat conductor from temperature data of some portion of the surface of the heat conductor. To apply the above stability results to this model problem, one needs the measurability and some boundedness of the forward operator. These properties are shown.
The efficiency of two Bayesian order estimators is studied. By using nonparametric techniques, we prove new underestimation and overestimation bounds. The results apply to various models, including mixture models. In this case, the errors are shown to be $O(e^{-an})$ and $O((\log n)^b/\sqrt{n})$ ($a,b>0$), respectively.
This paper compares classical parametric methods with recently developed Bayesian methods for system identification. A Full Bayes solution is considered together with one of the standard approximations based on the Empirical Bayes paradigm. Results regarding point estimators for the impulse response as well as for confidence regions are reported.
We consider the problem of modeling multivariate time series with parsimonious dynamical models which can be represented as sparse dynamic Bayesian networks with few latent nodes. This structure translates into a sparse plus low rank model. In this paper, we propose a Gaussian regression approach to identify such a model.
Model estimates obtained from traditional subspace identification methods may be subject to significant variance. This elevated variance is aggravated in the cases of large models or of a limited sample size. Common solutions to reduce the effect of variance are regularized estimators, shrinkage estimators and Bayesian estimation. In the current work we investigate the latter two solutions, which have not yet been applied to subspace identification. Our experimental results show that our proposed estimators may reduce the estimation risk up to $40\%$ of that of traditional subspace methods.
This paper presents a Bayesian method for identification of jump Markov linear system parameters. A primary motivation is to provide accurate quantification of parameter uncertainty without relying on asymptotic in data-length arguments. To achieve this, the paper details a particle-Gibbs sampling approach that provides samples from the desired posterior distribution. These samples are produced by utilising a modified discrete particle filter and carefully chosen conjugate priors.
System identification is of special interest in science and engineering. This article is concerned with a system identification problem arising in stochastic dynamic systems, where the aim is to estimate the parameters of a system along with its unknown noise processes. In particular, we propose a Bayesian nonparametric approach for system identification in discrete time nonlinear random dynamical systems assuming only the order of the Markov process is known. The proposed method replaces the assumption of Gaussian distributed error components with a highly flexible family of probability densi…
This paper describes the use of flexible Bayesian regression models for estimating a partially identified probability function. Our approach permits efficient sensitivity analysis concerning the posterior impact of priors on the partially identified component of the regression model. The new methodology is illustrated on an important problem where only partially observed data is available - inferring the prevalence of accounting misconduct among publicly traded U.S. businesses.
Harvey et al. (2010) extended the Bayesian estimation method by Sahu et al. (2003) to a multivariate skew-elliptical distribution with a general skewness matrix, and applied it to Bayesian portfolio optimization with higher moments. Although their method is epochal in the sense that it can handle the skewness dependency among asset returns and incorporate higher moments into portfolio optimization, it cannot identify all elements in the skewness matrix due to label switching in the Gibbs sampler. To deal with this identification issue, we propose to modify their sampling algorithm by imposing …
In this paper we develop a Bayesian statistical inference approach to the unified analysis of isobaric labelled MS/MS proteomic data across multiple experiments. An explicit probabilistic model of the log-intensity of the isobaric labels' reporter ions across multiple pre-defined groups and experiments is developed. This is then used to develop a full Bayesian statistical methodology for the identification of differentially expressed proteins, with respect to a control group, across multiple groups and experiments. This methodology is implemented and then evaluated on simulated data and on two…
In this paper, we propose an outlier-robust regularized kernel-based method for linear system identification. The unknown impulse response is modeled as a zero-mean Gaussian process whose covariance (kernel) is given by the recently proposed stable spline kernel, which encodes information on regularity and exponential stability. To build robustness to outliers, we model the measurement noise as realizations of independent Laplacian random variables. The identification problem is cast in a Bayesian framework, and solved by a new Markov Chain Monte Carlo (MCMC) scheme. In particular, exploiting …
In the last two decades, several methods based on sequential Monte Carlo (SMC) and Markov chain Monte Carlo (MCMC) have been proposed for Bayesian identification of stochastic non-linear state-space models (SSMs). It is well known that the performance of these simulation based identification methods depends on the numerical approximations used in their design. We propose the use of posterior Cramér-Rao lower bound (PCRLB) as a mean square error (MSE) bound. Using PCRLB, a systematic procedure is developed to analyse the estimates delivered by Bayesian identification methods in terms of bias, M…
The synthetic data approach to data confidentiality has been actively researched on, and for the past decade or so, a good number of high quality work on developing innovative synthesizers, creating appropriate utility measures and risk measures, among others, have been published. Comparing to a large volume of work on synthesizers development and utility measures creation, measuring risks has overall received less attention. This paper focuses on the detailed construction of some Bayesian methods proposed for estimating disclosure risks in synthetic data. In the processes of presenting attrib…
This paper presents a machine learning framework for Bayesian systems identification from noisy, sparse and irregular observations of nonlinear dynamical systems. The proposed method takes advantage of recent developments in differentiable programming to propagate gradient information through ordinary differential equation solvers and perform Bayesian inference with respect to unknown model parameters using Hamiltonian Monte Carlo. This allows us to efficiently infer posterior distributions over plausible models with quantified uncertainty, while the use of sparsity-promoting priors enables th…
Disease subtype identification (clustering) is an important problem in biomedical research. Gene expression profiles are commonly utilized to infer disease subtypes, which often lead to biologically meaningful insights into disease. Despite many successes, existing clustering methods may not perform well when genes are highly correlated and many uninformative genes are included for clustering due to the high dimensionality. In this article, we introduce a novel subtype identification method in the Bayesian setting based on gene expression profiles. This method, called BCSub, adopts an innovati…
We consider a Bayesian functional data analysis for observations measured as extremely long sequences. Splitting the sequence into a number of small windows with manageable length, the windows may not be independent especially when they are neighboring to each other. We propose to utilize Bayesian smoothing splines to estimate individual functional patterns within each window and to establish transition models for parameters involved in each window to address the dependent structure between windows. The functional difference of groups of individuals at each window can be evaluated by Bayes Fac…
Online Bayesian system identification in multivariate autoregressive models via message passing
Eindhoven Artificial Intelligence Systems Institute（埃因霍温人工智能系统研究所）；Department of Electrical Engineering, Eindhoven University of Technology（电子工程系，埃因霍温理工大学）
Bayesian model updating based on Gaussian Process (GP) models has received attention in recent years, which incorporates kernel-based GPs to provide enhanced fidelity response predictions. Although most kernel functions provide high fitting accuracy in the training data set, their out-of-sample predictions can be highly inaccurate. This paper investigates this problem by reformulating the problem on a consistent probabilistic foundation, reviewing common choices of kernel covariance functions, and proposing a new Bayesian model selection for kernel function selection, aiming to create a balanc…
The focus in this paper is Bayesian system identification based on noisy incomplete modal data where we can impose spatially-sparse stiffness changes when updating a structural model. To this end, based on a similar hierarchical sparse Bayesian learning model from our previous work, we propose two Gibbs sampling algorithms. The algorithms differ in their strategies to deal with the posterior uncertainty of the equation-error precision parameter, but both sample from the conditional posterior probability density functions (PDFs) for the structural stiffness parameters and system modal parameter…
We propose an algorithm for designing optimal inputs for on-line Bayesian identification of stochastic non-linear state-space models. The proposed method relies on minimization of the posterior Cramér Rao lower bound derived for the model parameters, with respect to the input sequence. To render the optimization problem computationally tractable, the inputs are parametrized as a multi-dimensional Markov chain in the input space. The proposed approach is illustrated through a simulation example.
We propose a flexible design for the identification of optimal dose combinations in dual-agent dose-finding clinical trials. The design is called AAA, standing for three adaptations: adaptive model selection, adaptive dose insertion, and adaptive cohort divi- sion. The adaptations highlight the need and opportunity for innovation for dual-agent dose finding, and are supported by the numerical results presented in the proposed simulation studies. To our knowledge, this is the first design that allows for all three adaptations at the same time. We find that AAA improves the statistical inference…
This study presents a Bayesian spectral density approach for identification and uncertainty quantification of flutter derivatives of bridge sections utilizing buffeting displacement responses, where the wind tunnel test is conducted in turbulent flow. Different from traditional time-domain approaches (e.g., least square method and stochastic subspace identification), the newly-proposed approach is operated in frequency domain. Based on the affine invariant ensemble sampler algorithm, Markov chain Monte-Carlo sampling is employed to accomplish the Bayesian inference. The probability density fun…
Structural identification and damage detection can be generalized as the simultaneous estimation of input forces, physical parameters, and dynamical states. Although Kalman-type filters are efficient tools to address this problem, the calibration of noise covariance matrices is cumbersome. For instance, calibration of input noise covariance matrix in augmented or dual Kalman filters is a critical task since a slight variation in its value can adversely affect estimations. The present study develops a Bayesian Expectation-Maximization (BEM) methodology for the uncertainty quantification and pro…
The increasing integration of intermittent renewable generation, especially at the distribution level,necessitates advanced planning and optimisation methodologies contingent on the knowledge of thegrid, specifically the admittance matrix capturing the topology and line parameters of an electricnetwork. However, a reliable estimate of the admittance matrix may either be missing or quicklybecome obsolete for temporally varying grids. In this work, we propose a data-driven identificationmethod utilising voltage and current measurements collected from micro-PMUs. More precisely,we first present a…
Massachusetts Institute of Technology（麻省理工学院）；Broad Institute of MIT and Harvard（MIT和哈佛大学Broad研究所）；Basis Research Institute（Basis研究机构）；Eric and Wendy Schmidt Center（埃里克和温迪·施密特中心）；BCAM (Basque Center for Applied Mathematics)（BCAM（巴斯克应用数学中心））；Ikerbasque (Basque Foundation for Science)（Ikerbasque（巴斯克科学基金会））
Rapidly assaying the diversity of a bacterial species present in a sample obtained from a hospital patient or an evironmental source has become possible after recent technological advances in DNA sequencing. For several applications it is important to accurately identify the presence and estimate relative abundances of the target organisms from short sequence reads obtained from a sample. This task is particularly challenging when the set of interest includes very closely related organisms, such as different strains of pathogenic bacteria, which can vary considerably in terms of virulence, res…
Inpatient care is a large share of total health care spending, making analysis of inpatient utilization patterns an important part of understanding what drives health care spending growth. Common features of inpatient utilization measures include zero inflation, over-dispersion, and skewness, all of which complicate statistical modeling. Mixture modeling is a popular approach that can accommodate these features of health care utilization data. In this work, we add a nonparametric clustering component to such models. Our fully Bayesian model framework allows for an unknown number of mixing comp…
Massachusetts Institute of Technology（麻省理工学院）；ExxonMobil Research and Engineering（埃克森美孚研究与工程公司）
In processing and manufacturing industries, there has been a large push to produce higher quality products and ensure maximum efficiency of processes. This requires approaches to effectively detect and resolve disturbances to ensure optimal operations. While the control system can compensate for many types of disturbances, there are changes to the process which it still cannot handle adequately. It is therefore important to further develop monitoring systems to effectively detect and identify those faults such that they can be quickly resolved by operators. In this paper, a novel probabilistic…
The present paper proposes a novel Bayesian, computational strategy in the context of model-based inverse problems in elastostatics. On one hand we attempt to provide probabilistic estimates of the material properties and their spatial variability that account for the various sources of uncertainty. On the other hand we attempt to address the question of model fidelity in relation to the experimental reality and particularly in the context of the material constitutive law adopted. This is especially important in biomedical settings when the inferred material properties will be used to make dec…
A Synthetic Reliability-Aware PINN Benchmark for Offshore Wind Turbine Support-Structure Monitoring with Bayesian Inverse Identification
School of Artificial Intelligence and Data Science, Indian Institute of Technology Jodhpur（印度理工学院焦特布尔分校人工智能与数据科学学院）；School of Management and Entrepreneurship, Indian Institute of Technology Jodhpur（印度理工学院焦特布尔分校管理与创业学院）
Sequential change diagnosis is the joint problem of detection and identification of a sudden and unobservable change in the distribution of a random sequence. In this problem, the common probability law of a sequence of i.i.d. random variables suddenly changes at some disorder time to one of finitely many alternatives. This disorder time marks the start of a new regime, whose fingerprint is the new law of observations. Both the disorder time and the identity of the new regime are unknown and unobservable. The objective is to detect the regime-change as soon as possible, and, at the same time, …
Environmental health researchers may aim to identify exposure patterns that represent sources, product use, or behaviors that give rise to mixtures of potentially harmful environmental chemical exposures. We present Bayesian non-parametric non-negative matrix factorization (BN^2MF) as a novel method to identify patterns of chemical exposures when the number of patterns is not known a priori. We placed non-negative continuous priors on pattern loadings and individual scores to enhance interpretability and used a clever non-parametric sparse prior to estimate the pattern number. We further deriv…
This work presents the estimation of the parameters of an experimental setup, which is modeled as a system with three degrees of freedom, composed by a shaft, two rotors, and a DC motor, that emulates a drilling process. A Bayesian technique is used in the estimation process, to take into account the uncertainties and variabilities intrinsic to the measurement taken, which are modeled as a noise of Gaussian nature. With this procedure it is expected to check the reliability of the nominal values of the physical parameters of the test rig. An estimation process assuming that nine parameters of …
The light we receive from distant astrophysical objects carries information about their origins and the physical mechanisms that power them. The study of these signals, however, is complicated by the fact that observations are often a mixture of the light emitted by multiple localized sources situated in a spatially-varying background. A general algorithm to achieve robust and accurate source identification in this case remains an open question in astrophysics. This paper focuses on high-energy light (such as X-rays and gamma-rays), for which observatories can detect individual photons (quanta…
State estimations and noise identifications with intermittent corrupted observations via Bayesian variational inference
School of Mathematical Sciences, Beihang University（北京航空航天大学数学科学学院）；Key Laboratory of Mathematics, Informatics and Behavioral Semantics (LMIB), Beihang University（北京航空航天大学数学、信息与行为语义重点实验室（LMIB））
Vector autoregressions (VARs) with multivariate stochastic volatility are widely used for structural analysis. Often the structural model identified through economically meaningful restrictions--e.g., sign restrictions--is supposed to be independent of how the dependent variables are ordered. But since the reduced-form model is not order invariant, results from the structural analysis depend on the order of the variables. We consider a VAR based on the factor stochastic volatility that is constructed to be order invariant. We show that the presence of multivariate stochastic volatility allows …
We consider the identification of large-scale linear and stable dynamic systems whose outputs may be the result of many correlated inputs. Hence, severe ill-conditioning may affect the estimation problem. This is a scenario often arising when modeling complex physical systems given by the interconnection of many sub-units where feedback and algebraic loops can be encountered. We develop a strategy based on Bayesian regularization where any impulse response is modeled as the realization of a zero-mean Gaussian process. The stable spline covariance is used to include information on smooth expone…
Tumor cells acquire different genetic alterations during the course of evolution in cancer patients. As a result of competition and selection, only a few subgroups of cells with distinct genotypes survive. These subgroups of cells are often referred to as subclones. In recent years, many statistical and computational methods have been developed to identify tumor subclones, leading to biologically significant discoveries and shedding light on tumor progression, metastasis, drug resistance and other processes. However, most existing methods are either not able to infer the phylogenetic structure…
A new Bayesian approach to linear system identification has been proposed in a series of recent papers. The main idea is to frame linear system identification as predictor estimation in an infinite dimensional space, with the aid of regularization/Bayesian techniques. This approach guarantees the identification of stable predictors based on the prediction error minimization. Unluckily, the stability of the predictors does not guarantee the stability of the impulse response of the system. In this paper we propose and compare various techniques to address this issue. Simulations results comparin…
The identification of accident hot spots is a central task of road safety management. Bayesian count data models have emerged as the workhorse method for producing probabilistic rankings of hazardous sites in road networks. Typically, these methods assume simple linear link function specifications, which, however, limit the predictive power of a model. Furthermore, extensive specification searches are precluded by complex model structures arising from the need to account for unobserved heterogeneity and spatial correlations. Modern machine learning (ML) methods offer ways to automate the speci…
