Bayesian recovery of the initial condition for the heat equation
B. T. Knapik, A. W. van der Vaart, J. H. van Zanten
备注：
17 pages, 4 figures. Published in Comm. Statist. Theory Methods. This version differs from the original in pagination and typographic detail. arXiv admin note: text overlap with arXiv:1103.2692
发表信息：
Communications in Statistics - Theory and Methods 2013 Volume 42, Issue 7, 1294-1313
We study a Bayesian approach to recovering the initial condition for the heat equation from noisy observations of the solution at a later time. We consider a class of prior distributions indexed by a parameter quantifying "smoothness" and show that the corresponding posterior distributions contract around the true parameter at a rate that depends on the smoothness of the true initial condition and the smoothness and scale of the prior. Correct combinations of these characteristics lead to the optimal minimax rate. One type of priors leads to a rate-adaptive Bayesian procedure. The frequentist coverage of credible sets is shown to depend on the combination of the prior and true parameter as well, with smoother priors leading to zero coverage and rougher priors to (extremely) conservative results. In the latter case credible sets are much larger than frequentist confidence sets, in that the ratio of diameters diverges to infinity. The results are numerically illustrated by a simulated data example.
Revealing underlying relations between nodes in a network is one of the most important tasks in network analysis. Using tools and techniques from a variety of disciplines, many community recovery methods have been developed for different scenarios. Despite the recent interest on community recovery in multilayer networks, theoretical results on the accuracy of the estimates are few and far between. Given a multilayer, e.g. temporal, network and a multilayer stochastic block model, we derive bounds for sufficient separation between intra- and inter-block connectivity parameters to achieve poster…
In this paper, we consider the phase recovery problem, where a complex signal vector has to be estimated from the knowledge of the modulus of its linear projections, from a naive variational Bayesian point of view. In particular, we derive an iterative algorithm following the minimization of the Kullback-Leibler divergence under the mean-field assumption, and show on synthetic data with random projections that this approach leads to an efficient and robust procedure, with a good computational cost.
Given data from a Poisson point process with intensity $(x,y) \mapsto n \mathbf{1}(f(x)\leq y),$ frequentist properties for the Bayesian reconstruction of the support boundary function $f$ are derived. We mainly study compound Poisson process priors with fixed intensity proving that the posterior contracts with nearly optimal rate for monotone and piecewise constant support boundaries and adapts to Hölder smooth boundaries with smoothness index at most one. We then derive a non-standard Bernstein-von Mises result for a compound Poisson process prior and a function space with increasing paramet…
University of Electronic Science and Technology of China（电子科技大学）；China Academy of Engineering Physics（中国工程物理研究院）
In this paper, we consider the block-sparse signals recovery problem in the context of multiple measurement vectors (MMV) with common row sparsity patterns. We develop a new method for recovery of common row sparsity MMV signals, where a pattern-coupled hierarchical Gaussian prior model is introduced to characterize both the block-sparsity of the coefficients and the statistical dependency between neighboring coefficients of the common row sparsity MMV signals. Unlike many other methods, the proposed method is able to automatically capture the block sparse structure of the unknown signal. Our …
University of Glasgow（格拉斯哥大学）；Amazon Research（亚马逊研究院）
We propose a new probabilistic method for unsupervised recovery of corrupted data. Given a large ensemble of degraded samples, our method recovers accurate posteriors of clean values, allowing the exploration of the manifold of possible reconstructed data and hence characterising the underlying uncertainty. In this setting, direct application of classical variational methods often gives rise to collapsed densities that do not adequately explore the solution space. Instead, we derive our novel reduced entropy condition approximate inference method that results in rich posteriors. We test our mo…
We propose a new, two-step empirical Bayes-type of approach for neural networks. We show in context of the nonparametric regression model that the procedure (up to a logarithmic factor) provides optimal recovery of the underlying functional parameter of interest and provides Bayesian credible sets with frequentist coverage guarantees. The approach requires fitting the neural network only once, hence it is substantially faster than Bootstrapping type approaches. We demonstrate the applicability of our method over synthetic data, observing good estimation properties and reliable uncertainty quan…
We address the sparse signal recovery problem in the context of multiple measurement vectors (MMV) when elements in each nonzero row of the solution matrix are temporally correlated. Existing algorithms do not consider such temporal correlations and thus their performance degrades significantly with the correlations. In this work, we propose a block sparse Bayesian learning framework which models the temporal correlations. In this framework we derive two sparse Bayesian learning (SBL) algorithms, which have superior recovery performance compared to existing algorithms, especially in the presen…
In this paper, we propose a generalized scale mixture family of distributions, namely the Power Exponential Scale Mixture (PESM) family, to model the sparsity inducing priors currently in use for sparse signal recovery (SSR). We show that the successful and popular methods such as LASSO, Reweighted $\ell_1$ and Reweighted $\ell_2$ methods can be formulated in an unified manner in a maximum a posteriori (MAP) or Type I Bayesian framework using an appropriate member of the PESM family as the sparsity inducing prior. In addition, exploiting the natural hierarchical framework induced by the PESM f…
Max Planck Institute for Molecular Genetics（马克斯·普朗克分子遗传学研究所）
We present a Bayesian method for feature selection in the presence of grouping information with sparsity on the between- and within group level. Instead of using a stochastic algorithm for parameter inference, we employ expectation propagation, which is a deterministic and fast algorithm. Available methods for feature selection in the presence of grouping information have a number of short-comings: on one hand, lasso methods, while being fast, underestimate the regression coefficients and do not make good use of the grouping information, and on the other hand, Bayesian approaches, while accura…
Clipping is one of the simplest peak-to-average power ratio (PAPR) reduction schemes for orthogonal frequency division multiplexing (OFDM). Deliberately clipping the transmission signal degrades system performance, and clipping mitigation is required at the receiver for information restoration. In this work, we acknowledge the sparse nature of the clipping signal and propose a low-complexity Bayesian clipping estimation scheme. The proposed scheme utilizes a priori information about the sparsity rate and noise variance for enhanced recovery. At the same time, the proposed scheme is robust agai…
Sparse signal reconstruction algorithms have attracted research attention due to their wide applications in various fields. In this paper, we present a simple Bayesian approach that utilizes the sparsity constraint and a priori statistical information (Gaussian or otherwise) to obtain near optimal estimates. In addition, we make use of the rich structure of the sensing matrix encountered in many signal processing applications to develop a fast sparse recovery algorithm. The computational complexity of the proposed algorithm is relatively low compared with the widely used convex relaxation meth…
University of Electronic Science and Technology of China（电子科技大学）；Stevens Institute of Technology（史蒂文斯理工学院）；Schlumberger-Doll Research Center（斯伦贝谢多尔研究中心）
We consider the problem of recovering block-sparse signals whose structures are unknown \emph{a priori}. Block-sparse signals with nonzero coefficients occurring in clusters arise naturally in many practical scenarios. However, the knowledge of the block structure is usually unavailable in practice. In this paper, we develop a new sparse Bayesian learning method for recovery of block-sparse signals with unknown cluster patterns. Specifically, a pattern-coupled hierarchical Gaussian prior model is introduced to characterize the statistical dependencies among coefficients, in which a set of hype…
The application of compressive sensing (CS) to structural health monitoring is an emerging research topic. The basic idea in CS is to use a specially-designed wireless sensor to sample signals that are sparse in some basis (e.g. wavelet basis) directly in a compressed form, and then to reconstruct (decompress) these signals accurately using some inversion algorithm after transmission to a central processing unit. However, most signals in structural health monitoring are only approximately sparse, i.e. only a relatively small number of the signal coefficients in some basis are significant, but …
A general shape identification inverse problem is studied in a Bayesian framework. This problem requires the determination of the unknown shape of a domain in the Euclidean space from finite-dimensional observation data with some Gaussian random noise. Then, the stability of posterior is studied for observation data. For each point of the space, the conditional probability that the point is included in the unknown domain given the observation data is considered. The stability is also studied for this probability distribution. As a model problem for our inverse problem, a heat inverse problem i…
Bayesian regularization is a central tool in modern-day statistical and machine learning methods. Many applications involve high-dimensional sparse signal recovery problems. The goal of our paper is to provide a review of the literature on penalty-based regularization approaches, from Tikhonov (Ridge, Lasso) to horseshoe regularization.
National University of Defense Technology（国防科技大学）；Samsung R&D Institute America - Dallas（三星美国研究院（达拉斯））
The performance of sparse signal recovery from noise corrupted, underdetermined measurements can be improved if both sparsity and correlation structure of signals are exploited. One typical correlation structure is the intra-block correlation in block sparse signals. To exploit this structure, a framework, called block sparse Bayesian learning (BSBL), has been proposed recently. Algorithms derived from this framework showed superior performance but they are not very fast, which limits their applications. This work derives an efficient algorithm from this framework, using a marginalized likelih…
We consider the problem of uncertainty quantification for an unknown low-rank matrix $\mathbf{X}$, given a partial and noisy observation of its entries. This quantification of uncertainty is essential for many real-world problems, including image processing, satellite imaging, and seismology, providing a principled framework for validating scientific conclusions and guiding decision-making. However, existing literature has mainly focused on the completion (i.e., point estimation) of the matrix $\mathbf{X}$, with little work on investigating its uncertainty. To this end, we propose in this work…
We propose a Bayesian model that predicts recovery curves based on information available before the disruptive event. A recovery curve of interest is the quantified sexual function of prostate cancer patients after prostatectomy surgery. We illustrate the utility of our model as a pre-treatment medical decision aid, producing personalized predictions that are both interpretable and accurate. We uncover covariate relationships that agree with and supplement that in existing medical literature.
The accurate quantification of changes in the heat content of the world's oceans is crucial for our understanding of the effects of increasing greenhouse gas concentrations. The Argo program, consisting of Lagrangian floats that measure vertical temperature profiles throughout the global ocean, has provided a wealth of data from which to estimate ocean heat content. However, creating a globally consistent statistical model for ocean heat content remains challenging due to the need for a globally valid covariance model that can capture complex nonstationarity. In this paper, we develop a hierar…
Quantum state tomography, an important task in quantum information processing, aims at reconstructing a state from prepared measurement data. Bayesian methods are recognized to be one of the good and reliable choice in estimating quantum states~\cite{blume2010optimal}. Several numerical works showed that Bayesian estimations are comparable to, and even better than other methods in the problem of $1$-qubit state recovery. However, the problem of choosing prior distribution in the general case of $n$ qubits is not straightforward. More importantly, the statistical performance of Bayesian type es…
Bayesian networks are convenient graphical expressions for high dimensional probability distributions representing complex relationships between a large number of random variables. They have been employed extensively in areas such as bioinformatics, artificial intelligence, diagnosis, and risk management. The recovery of the structure of a network from data is of prime importance for the purposes of modeling, analysis, and prediction. Most recovery algorithms in the literature assume either discrete of continuous but Gaussian data. For general continuous data, discretization is usually employe…
The assessment of the thermal properties of walls is essential for accurate building energy simulations that are needed to make effective energy-saving policies. These properties are usually investigated through in-situ measurements of temperature and heat flux over extended time periods. The one-dimensional heat equation with unknown Dirichlet boundary conditions is used to model the heat transfer process through the wall. In [F. Ruggeri, Z. Sawlan, M. Scavino, R. Tempone, A hierarchical Bayesian setting for an inverse problem in linear parabolic PDEs with noisy boundary conditions, Bayesian …
A common approach to assess the performance of fire insulation panels is the component additive method (CAM). The parameters of the CAM are based on the temperature-dependent thermal material properties of the panels. These material properties can be derived by calibrating finite element heat transfer models using experimentally measured temperature records. In the past, the calibration of the material properties was done manually by trial and error approaches, which was inefficient and prone to error. In this contribution, the calibration problem is reformulated in a probabilistic setting and…
University Paris-Est（巴黎东大学）；IBM Research China（IBM中国研究院）；CentraleSupelec（中央理工-高等电力学院）；University Paris-Saclay（巴黎萨克雷大学）
In this paper, a methodology is investigated for signal recovery in the presence of non-Gaussian noise. In contrast with regularized minimization approaches often adopted in the literature, in our algorithm the regularization parameter is reliably estimated from the observations. As the posterior density of the unknown parameters is analytically intractable, the estimation problem is derived in a variational Bayesian framework where the goal is to provide a good approximation to the posterior distribution in order to compute posterior mean estimates. Moreover, a majorization technique is emplo…
Exponential family models, generalized linear models (GLMs), generalized linear mixed models (GLMMs) and generalized additive models (GAMs) are widely used methods in statistics. However, many scientific applications necessitate constraints be placed on model parameters such as shape and linear inequality constraints. Constrained estimation and inference of parameters remains a pervasive problem in statistics where many methods rely on modifying rigid large sample theory assumptions for inference. We propose a flexible slice sampler Gibbs algorithm for Bayesian GLMMs and GAMs with linear inequ…
University of Oxford（牛津大学）；Technische Universität Darmstadt（达姆施塔特工业大学）
Unstructured data from diverse sources, such as social media and aerial imagery, can provide valuable up-to-date information for intelligent situation assessment. Mining these different information sources could bring major benefits to applications such as situation awareness in disaster zones and mapping the spread of diseases. Such applications depend on classifying the situation across a region of interest, which can be depicted as a spatial "heatmap". Annotating unstructured data using crowdsourcing or automated classifiers produces individual classifications at sparse locations that typic…
University of California, San Diego（加州大学圣地亚哥分校）；Microsoft Research（微软研究院）
This work discusses the problem of sparse signal recovery when there is correlation among the values of non-zero entries. We examine intra-vector correlation in the context of the block sparse model and inter-vector correlation in the context of the multiple measurement vector model, as well as their combination. Algorithms based on the sparse Bayesian learning are presented and the benefits of incorporating correlation at the algorithm level are discussed. The impact of correlation on the limits of support recovery is also discussed highlighting the different impact intra-vector and inter-vec…
We introduce a computational framework to statistically infer thermophysical properties of any given wall from in-situ measurements of air temperature and surface heat fluxes. The proposed framework uses these measurements, within a Bayesian calibration approach, to sequentially infer input parameters of a one-dimensional heat diffusion model that describes the thermal performance of the wall. These inputs include spatially-variable functions that characterise the thermal conductivity and the volumetric heat capacity of the wall. We encode our computational framework in an algorithm that seque…
Matrix and tensor completion are frameworks for a wide range of problems, including collaborative filtering, missing data, and image reconstruction. Missing entries are estimated by leveraging an assumption that the matrix or tensor is low-rank. Most existing Bayesian techniques encourage rank-sparsity by modelling factorized matrices and tensors with Normal-Gamma priors. However, the Horseshoe prior and other "global-local" formulations provide tuning-parameter-free solutions which may better achieve simultaneous rank-sparsity and missing-value recovery. We find these global-local priors outp…
The hydroxyl radical is the primary reactive oxygen species produced by the radiolysis of water, and is a significant source of radiation damage to living organisms. Mobility of the hydroxyl radical at low temperatures and/or high pressures is hence a potentially important factor in determining the challenges facing psychrophilic and/or barophilic organisms in high-radiation environments (e.g., ice-interface or undersea environments in which radiative heating is a potential heat and energy source). Here, we estimate the diffusion coefficient for the hydroxyl radical in aqueous solution, using …
Iterative reweighted algorithms, as a class of algorithms for sparse signal recovery, have been found to have better performance than their non-reweighted counterparts. However, for solving the problem of multiple measurement vectors (MMVs), all the existing reweighted algorithms do not account for temporal correlation among source vectors and thus their performance degrades significantly in the presence of correlation. In this work we propose an iterative reweighted sparse Bayesian learning (SBL) algorithm exploiting the temporal correlation, and motivated by it, we propose a strategy to impr…
We address the inverse problem for the mean-field Ising model with two- and three-body interactions using a Bayesian framework. Parameter recovery in this setting is notoriously difficult, particularly near phase transitions, at criticality, and under non-identifiability, where conventional estimators and standard MCMC samplers fail. To overcome these challenges, we develop a hybrid algorithm that combines Adaptive Metropolis Hastings with geometry-aware Riemannian manifold Hamiltonian dynamics. This approach yields substantially improved mixing and convergence in the three-dimensional paramet…
We examine the recovery of block sparse signals and extend the framework in two important directions; one by exploiting signals' intra-block correlation and the other by generalizing signals' block structure. We propose two families of algorithms based on the framework of block sparse Bayesian learning (BSBL). One family, directly derived from the BSBL framework, requires knowledge of the block structure. Another family, derived from an expanded BSBL framework, is based on a weaker assumption on the block structure, and can be used when the block structure is completely unknown. Using these al…
A fast matching pursuit method using a Bayesian approach is introduced for sparse signal recovery. This method, referred to as nGpFBMP, performs Bayesian estimates of sparse signals even when the signal prior is non-Gaussian or unknown. It is agnostic on signal statistics and utilizes a priori statistics of additive noise and the sparsity rate of the signal, which are shown to be easily estimated from data if not available. nGpFBMP utilizes a greedy approach and order-recursive updates of its metrics to find the most dominant sparse supports to determine the approximate minimum mean square err…
University College London（伦敦大学学院）；Max Planck UCL Centre for Computational Psychiatry and Ageing Research（马克斯·普朗克伦敦大学学院计算精神病学与老龄化研究中心）；Institute of Cognitive Neuroscience（认知神经科学研究所）
Hierarchical model fitting has become commonplace for case-control studies of cognition and behaviour in mental health. However, these techniques require us to formalise assumptions about the data-generating process at the group level, which may not be known. Specifically, researchers typically must choose whether to assume all subjects are drawn from a common population, or to model them as deriving from separate populations. These assumptions have profound implications for computational psychiatry, as they affect the resulting inference (latent parameter recovery) and may conflate or mask tr…
