This paper reviews how a diverse set of popular data-driven priors commonly used in Bayesian inverse problems can be unified through their respective score functions. By framing these priors under this common perspective, we show that they can benefit from their straightfoward and effective integration into a recently proposed sampling algorithm. The applicability of this common framework is illustrated by considering several data-driven priors, namely regularization-by-denoising, normalizing flow-based priors, score-based generative models, and convex-ridge regularizers. For these four particular priors, the performance of the method is evaluated when conducting image inpainting and single image super-resolution. These results, as well as those obtained when restoring real images acquired in a geological context, demonstrate the efficiency of the method. This unified framework proves versatile enough to handle any posterior distribution defined by a broad class of score function-based priors, beyond the specific cases considered in this paper.
We present a new Markov chain Monte Carlo algorithm, implemented in software Arbores, for inferring the history of a sample of DNA sequences. Our principal innovation is a bridging procedure, previously applied only for simple stochastic processes, in which the local computations within a bridge can proceed independently of the rest of the DNA sequence, facilitating large-scale parallelisation.
One of the challenges in training generative models such as the variational auto encoder (VAE) is avoiding posterior collapse. When the generator has too much capacity, it is prone to ignoring latent code. This problem is exacerbated when the dataset is small, and the latent dimension is high. The root of the problem is the ELBO objective, specifically the Kullback-Leibler (KL) divergence term in objective function \citep{zhao2019infovae}. This paper proposes a new objective function to replace the KL term with one that emulates the maximum mean discrepancy (MMD) objective. It also introduces …
NEC Corporation（日本电气股份有限公司）；National Institute of Advanced Industrial Science and Technology（产业技术综合研究所）
The interpretability of machine learning, particularly for deep neural networks, is crucial for decision making in real-world applications. One approach is replacing the un-interpretable machine learning model with a surrogate model, which has a simple structure for interpretation. Another approach is understanding the target system by using a simulation modeled by human knowledge with interpretable simulation parameters. Recently, simulator calibration has been developed based on kernel mean embedding to estimate the simulation parameters as posterior distributions. Our idea is to use a simul…
University of Oxford（牛津大学）；Novo Nordisk（诺和诺德）；University College London（伦敦大学学院）
Bayesian methods are a popular choice for statistical inference in small-data regimes due to the regularization effect induced by the prior. In the context of density estimation, the standard nonparametric Bayesian approach is to target the posterior predictive of the Dirichlet process mixture model. In general, direct estimation of the posterior predictive is intractable and so methods typically resort to approximating the posterior distribution as an intermediate step. The recent development of quasi-Bayesian predictive copula updates, however, has made it possible to perform tractable predi…
Several structure-learning algorithms for staged trees, asymmetric extensions of Bayesian networks, have been proposed. However, these either do not scale efficiently as the number of variables considered increases, a priori restrict the set of models, or they do not find comparable models to existing methods. Here, we define an alternative algorithm based on a totally ordered hyperstage. We demonstrate how it can be used to obtain a quadratically-scaling structural learning algorithm for staged trees that restricts the model space a-posteriori. Through comparative analysis, we show that throu…
University of Cambridge（剑桥大学）；IT University of Copenhagen（哥本哈根信息技术大学）
In this paper, we study the trade-offs of different inference approaches for Bayesian matrix factorisation methods, which are commonly used for predicting missing values, and for finding patterns in the data. In particular, we consider Bayesian nonnegative variants of matrix factorisation and tri-factorisation, and compare non-probabilistic inference, Gibbs sampling, variational Bayesian inference, and a maximum-a-posteriori approach. The variational approach is new for the Bayesian nonnegative models. We compare their convergence, and robustness to noise and sparsity of the data, on both synt…
In segmentation problems, inference on change-point position and model selection are two difficult issues due to the discrete nature of change-points. In a Bayesian context, we derive exact, non-asymptotic, explicit and tractable formulae for the posterior distribution of variables such as the number of change-points or their positions. We also derive a new selection criterion that accounts for the reliability of the results. All these results are based on an efficient strategy to explore the whole segmentation space, which is very large. We illustrate our methodology on both simulated data an…
The first-order autoregressive process, AR (1), has been widely used and implemented in time series analysis. Different estimation methods have been employed in order to estimate the autoregressive parameter. This article focuses on subjective Bayesian estimation as opposed to objective Bayesian estimation and frequentist procedures. The truncated normal distribution is considered as a prior, to impose stationarity. The posterior distribution as well as the Bayes estimator are derived. A comparative study between the newly derived estimator and other existing estimation methods (frequentist) i…
The identification of parameters in mathematical models using noisy observations is a common task in uncertainty quantification. We employ the framework of Bayesian inversion: we combine monitoring and observational data with prior information to estimate the posterior distribution of a parameter. Specifically, we are interested in the distribution of a diffusion coefficient of an elliptic PDE. In this setting, the sample space is high-dimensional, and each sample of the PDE solution is expensive. To address these issues we propose and analyse a novel Sequential Monte Carlo (SMC) sampler for t…
Bridging Data Barriers among Participants: Assessing the Potential of Geoenergy through Federated Learning
Ningbo Institute of Digital Twin, Eastern Institute of Technology（宁波数字孪生研究院，东方理工大学）；The Hong Kong Polytechnic University（香港理工大学）；Shanghai Jiao Tong University（上海交通大学）；Research Institute for Smart Energy, The Hong Kong Polytechnic University（香港理工大学智慧能源研究院）
D-Flow SGLD: Source-Space Posterior Sampling for Scientific Inverse Problems with Flow Matching
Sibley School of Mechanical and Aerospace Engineering, Cornell University（科尼尔大学机械与航空航天工程学院）；Department of Aerospace and Mechanical Engineering, University of Notre Dame（诺丁汉大学航空航天与机械工程系）
Social and economic studies are often implemented as complex survey designs. For example, multistage, unequal probability sampling designs utilized by federal statistical agencies are typically constructed to maximize the efficiency of the target domain level estimator (e.g., indexed by geographic area) within cost constraints for survey administration. Such designs may induce dependence between the sampled units; for example, with employment of a sampling step that selects geographically-indexed clusters of units. A sampling-weighted pseudo-posterior distribution may be used to estimate the p…
Misspecification-robust amortised simulation-based inference using variational methods
Institute of Astronomy, University of Cambridge（天文研究所，剑桥大学）；Statistical Laboratory, University of Cambridge（统计实验室，剑桥大学）；Kavli Institute for Cosmology, University of Cambridge（宇宙学研究所，剑桥大学）；Institute of Astrophysics, FORTH（天体物理学研究所，福蒂斯研究所）
Uppsala University（乌普萨拉大学）；University of California-Santa Barbara（加州大学圣塔芭芭拉分校）；Umeå University（于默奥大学）
State-of-the-art neural network-based methods for learning summary statistics have delivered promising results for simulation-based likelihood-free parameter inference. Existing approaches require density estimation as a post-processing step building upon deterministic neural networks, and do not take network prediction uncertainty into account. This work proposes a robust integrated approach that learns summary statistics using Bayesian neural networks, and directly estimates the posterior density using categorical distributions. An adaptive sampling scheme selects simulation locations to eff…
Variational inference provides approximations to the computationally intractable posterior distribution in Bayesian networks. A prominent medical application of noisy-or Bayesian network is to infer potential diseases given observed symptoms. Previous studies focus on approximating a handful of complicated pathological cases using variational transformation. Our goal is to use variational transformation as part of a novel hybridized inference for serving reliable and real time diagnosis at web scale. We propose a hybridized inference that allows variational parameters to be estimated without d…
Thermodynamic integration (TI) for computing marginal likelihoods is based on an inverse annealing path from the prior to the posterior distribution. In many cases, the resulting estimator suffers from high variability, which particularly stems from the prior regime. When comparing complex models with differences in a comparatively small number of parameters, intrinsic errors from sampling fluctuations may outweigh the differences in the log marginal likelihood estimates. In the present article, we propose a thermodynamic integration scheme that directly targets the log Bayes factor. The metho…
Institute of Automation, Chinese Academy of Sciences（中国科学院自动化研究所）；School of Artificial Intelligence, University of Chinese Academy of Sciences（中国科学院大学人工智能学院）
Variational autoencoders (VAEs), as an important aspect of generative models, have received a lot of research interests and reached many successful applications. However, it is always a challenge to achieve the consistency between the learned latent distribution and the prior latent distribution when optimizing the evidence lower bound (ELBO), and finally leads to an unsatisfactory performance in data generation. In this paper, we propose a latent distribution consistency approach to avoid such substantial inconsistency between the posterior and prior latent distributions in ELBO optimizing. W…
Bayesian model criticism is an important part of the practice of Bayesian statistics. Traditionally, model criticism methods have been based on the predictive check, an adaptation of goodness-of-fit testing to Bayesian modeling and an effective method to understand how well a model captures the distribution of the data. In modern practice, however, researchers iteratively build and develop many models, exploring a space of models to help solve the problem at hand. While classical predictive checks can help assess each one, they cannot help the researcher understand how the models relate to eac…
Class-conditional extensions of generative adversarial networks (GANs), such as auxiliary classifier GAN (AC-GAN) and conditional GAN (cGAN), have garnered attention owing to their ability to decompose representations into class labels and other factors and to boost the training stability. However, a limitation is that they assume that each class is separable and ignore the relationship between classes even though class overlapping frequently occurs in a real-world scenario when data are collected on the basis of diverse or ambiguous criteria. To overcome this limitation, we address a novel pr…
We propose a data-driven, coarse-graining formulation in the context of equilibrium statistical mechanics. In contrast to existing techniques which are based on a fine-to-coarse map, we adopt the opposite strategy by prescribing a probabilistic coarse-to-fine map. This corresponds to a directed probabilistic model where the coarse variables play the role of latent generators of the fine scale (all-atom) data. From an information-theoretic perspective, the framework proposed provides an improvement upon the relative entropy method and is capable of quantifying the uncertainty due to the informa…
Generative Adversarial Networks Bridging Art and Machine Intelligence
Imperial College London（帝国理工学院）；The University of Texas at Dallas（德克萨斯大学达拉斯分校）；Indiana University（印第安纳大学）；Xi’an Jiaotong-Liverpool University（西交利物浦大学）；Georgia Institute of Technology（佐治亚理工学院）；Kyoto University（京都大学）；AppCubic；Rutgers University（罗格斯大学）；Purdue University（普渡大学）；University of Wisconsin-Madison（威斯康星大学麦迪逊分校）；National Taiwan Normal University（台湾师范大学）；University of Hawaii（夏威夷大学）；Hong Kong University of Science and Technology（香港科技大学）；School of Visual Arts（视觉艺术学院）；Emory University（埃默里大学）；Aarhus University（奥胡斯大学）；Zhejiang University（浙江大学）；University of Edinburgh（爱丁堡大学）；National Tsing Hua University（国立清华大学）；University of Manchester（曼彻斯特大学）；Pingtan Research Institute of Xiamen University（厦门大学平潭研究院）；University of Liverpool（利物浦大学）；JTB Technology Corp.（JTB科技公司）
This article presents a comparative study of three different types of estimators used for supervised linear unmixing of two MEx/OMEGA hyperspectral cubes. The algorithms take into account the constraints of the abundance fractions, in order to get physically interpretable results. Abundance maps show that the Bayesian maximum a posteriori probability (MAP) estimator proposed in Themelis and Rontogiannis (2008) outperforms the other two schemes, offering a compromise between complexity and estimation performance. Thus, the MAP estimator is a candidate algorithm to perform ice and minerals detec…
Multi-dimensional functional data arises in numerous modern scientific experimental and observational studies. In this paper we focus on longitudinal functional data, a structured form of multidimensional functional data. Operating within a longitudinal functional framework we aim to capture low dimensional interpretable features. We propose a computationally efficient nonparametric Bayesian method to simultaneously smooth observed data, estimate conditional functional means and functional covariance surfaces. Statistical inference is based on Monte Carlo samples from the posterior measure thr…
A two-groups mixed-effects model for the comparison of (normalized) microarray data from two treatment groups is considered. Most competing parametric methods that have appeared in the literature are obtained as special cases or by minor modification of the proposed model. Approximate maximum likelihood fitting is accomplished via a fast and scalable algorithm, which we call LEMMA (Laplace approximated EM Microarray Analysis). The posterior odds of treatment $\times$ gene interactions, derived from the model, involve shrinkage estimates of both the interactions and of the gene specific error v…
This is an overview of the R package iprior, which implements a unified methodology for fitting parametric and nonparametric regression models, including additive models, multilevel models, and models with one or more functional covariates. Based on the principle of maximum entropy, an I-prior is an objective Gaussian process prior for the regression function with covariance kernel equal to its Fisher information. The regression function is estimated by its posterior mean under the I-prior, and hyperparameters are estimated via maximum marginal likelihood. Estimation of I-prior models is simpl…
