In this work, we propose a new Gaussian process regression (GPR) method: physics information aided Kriging (PhIK). In the standard data-driven Kriging, the unknown function of interest is usually treated as a Gaussian process with assumed stationary covariance with hyperparameters estimated from data. In PhIK, we compute the mean and covariance function from realizations of available stochastic models, e.g., from realizations of governing stochastic partial differential equations solutions. Such constructed Gaussian process generally is non-stationary, and does not assume a specific form of the covariance function. Our approach avoids the optimization step in data-driven GPR methods to identify the hyperparameters. More importantly, we prove that the physical constraints in the form of a deterministic linear operator are guaranteed in the resulting prediction. We also provide an error estimate in preserving the physical constraints when errors are included in the stochastic model realizations. To reduce the computational cost of obtaining stochastic model realizations, we propose a multilevel Monte Carlo estimate of the mean and covariance functions. Further, we present an active learning algorithm that guides the selection of additional observation locations. The efficiency and accuracy of PhIK are demonstrated for reconstructing a partially known modified Branin function, studying a three-dimensional heat transfer problem and learning a conservative tracer distribution from sparse concentration measurements.
University of California, Los Angeles（加利福尼亚大学洛杉矶分校）；University of Colorado Boulder（科罗拉多大学博尔德分校）
Treeging combines the flexible mean structure of regression trees with the covariance-based prediction strategy of kriging into the base learner of an ensemble prediction algorithm. In so doing, it combines the strengths of the two primary types of spatial and space-time prediction models: (1) models with flexible mean structures (often machine learning algorithms) that assume independently distributed data, and (2) kriging or Gaussian Process (GP) prediction models with rich covariance structures but simple mean structures. We investigate the predictive accuracy of treeging across a thorough …
Gaussian process models -also called Kriging models- are often used as mathematical approximations of expensive experiments. However, the number of observation required for building an emulator becomes unrealistic when using classical covariance kernels when the dimension of input increases. In oder to get round the curse of dimensionality, a popular approach is to consider simplified models such as additive models. The ambition of the present work is to give an insight into covariance kernels that are well suited for building additive Kriging models and to describe some properties of the resu…
In the context of Gaussian Process Regression or Kriging, we propose a full-Bayesian solution to deal with hyperparameters of the covariance function. This solution can be extended to the Trans-Gaussian Kriging framework, which makes it possible to deal with spatial data sets that violate assumptions required for Kriging. It is shown to be both elegant and efficient. We propose an application to computer experiments in the field of nuclear safety, where it is necessary to model non-destructive testing procedures based on eddy currents to detect possible wear in steam generator tubes.
Kriging based on Gaussian random fields is widely used in reconstructing unknown functions. The kriging method has pointwise predictive distributions which are computationally simple. However, in many applications one would like to predict for a range of untried points simultaneously. In this work we obtain some error bounds for the (simple) kriging predictor under the uniform metric. It works for a scattered set of input points in an arbitrary dimension, and also covers the case where the covariance function of the Gaussian process is misspecified. These results lead to a better understanding…
Institut für Baumechanik und Numerische Mechanik（建筑力学与数值力学研究所）
The computational effort for the evaluation of numerical simulations based on e.g. the finite-element method is high. Metamodels can be utilized to create a low-cost alternative. However the number of required samples for the creation of a sufficient metamodel should be kept low, which can be achieved by using adaptive sampling techniques. In this Master thesis adaptive sampling techniques are investigated for their use in creating metamodels with the Kriging technique, which interpolates values by a Gaussian process governed by prior covariances. The Kriging framework with extension to multif…
Informative sampling designs can impact spatial prediction, or kriging, in two important ways. First, the sampling design can bias spatial covariance parameter estimation, which in turn can bias spatial kriging estimates. Second, even with unbiased estimates of the spatial covariance parameters, since the kriging variance is a function of the observation locations, these estimates will vary based on the sample and overestimate the population-based estimates. In this work, we develop a weighted composite likelihood approach to improve spatial covariance parameter estimation under informative sa…
East China Normal University（华东师范大学）；King Abdullah University of Science and Technology（阿卜杜拉国王科技大学）；North Carolina State University（北卡罗来纳州立大学）
In spatial statistics, a common objective is to predict values of a spatial process at unobserved locations by exploiting spatial dependence. Kriging provides the best linear unbiased predictor using covariance functions and is often associated with Gaussian processes. However, when considering non-linear prediction for non-Gaussian and categorical data, the Kriging prediction is no longer optimal, and the associated variance is often overly optimistic. Although deep neural networks (DNNs) are widely used for general classification and prediction, they have not been studied thoroughly for data…
Florida State University（佛罗里达州立大学）；Northwestern University（西北大学）
This paper presents a new approach for Gaussian process (GP) regression for large datasets. The approach involves partitioning the regression input domain into multiple local regions with a different local GP model fitted in each region. Unlike existing local partitioned GP approaches, we introduce a technique for patching together the local GP models nearly seamlessly to ensure that the local GP models for two neighboring regions produce nearly the same response prediction and prediction error variance on the boundary between the two regions. This largely mitigates the well-known discontinuit…
Gaussian process models typically contain finite dimensional parameters in the covariance function that need to be estimated from the data. We study the Bayesian fixed-domain asymptotics for the covariance parameters in a universal kriging model with an isotropic Matern covariance function, which has many applications in spatial statistics. We show that when the dimension of domain is less than or equal to three, the joint posterior distribution of the microergodic parameter and the range parameter can be factored independently into the product of their marginal posteriors under fixed-domain a…
Remote sensing data products often include quality flags that inform users whether the associated observations are of good, acceptable or unreliable qualities. However, such information on data fidelity is not considered in remote sensing data analyses. Motivated by observations from the Atmospheric Infrared Sounder (AIRS) instrument on board NASA's Aqua satellite, we propose a latent variable co-kriging model with separable Gaussian processes to analyze large quality-flagged remote sensing data sets together with their associated quality information. We augment the posterior distribution by a…
Spatial prediction requires expensive computation to invert the spatial covariance matrix it depends on and also has considerable storage needs. This work concentrates on computationally efficient algorithms for prediction using very large datasets. A recent prediction model for spatial data known as Fixed Rank Kriging is much faster than the kriging and can be easily implemented with less assumptions about the process. However, Fixed Rank Kriging requires the estimation of a matrix which must be positive definite and the original estimation procedure cannot guarantee this property. We present…
When is multivariate kriging worthwhile? Design geometry, parameter sharing and response coupling under heterotopic sampling
University of Edinburgh Business School（爱丁堡大学商学院）；Faculty of Science and Engineering, University of Nottingham Ningbo China（宁波诺丁汉大学理学院）；College of Data Science, Jiaxing University（嘉兴大学数据科学学院）
This work is focused on finding G-optimal designs theoretically for kriging models with two-dimensional inputs and separable exponential covariance structures. For design comparison, the notion of evenness of two-dimensional grid designs is developed. The mathematical relationship between the design and the supremum of the mean squared prediction error ($SMSPE$) function is studied and then optimal designs are explored for both prospective and retrospective design scenarios. In the case of prospective designs, the new design is developed before the experiment is conducted and the regularly spa…
Université de Lille（里尔大学）；CNRS/CRIStAL（法国国家科学研究中心/CRIStAL实验室）；Inria Lille（法国国家信息与自动化研究所里尔中心）
Efficient Global Optimization (EGO) is widely used for the optimization of computationally expensive black-box functions. It uses a surrogate modeling technique based on Gaussian Processes (Kriging). However, due to the use of a stationary covariance, Kriging is not well suited for approximating non stationary functions. This paper explores the integration of Deep Gaussian processes (DGP) in EGO framework to deal with the non-stationary issues and investigates the induced challenges and opportunities. Numerical experimentations are performed on analytical problems to highlight the different as…
We propose a three-step divide-and-conquer strategy within the Bayesian paradigm that delivers massive scalability for any spatial process model. We partition the data into a large number of subsets, apply a readily available Bayesian spatial process model on every subset, in parallel, and optimally combine the posterior distributions estimated across all the subsets into a pseudo-posterior distribution that conditions on the entire data. The combined pseudo posterior distribution replaces the full data posterior distribution for predicting the responses at arbitrary locations and for inferenc…
Covariance tapering is a popular approach for reducing the computational cost of spatial prediction and parameter estimation for Gaussian process models. However, tapering can have poor performance when the process is sampled at spatially irregular locations or when non-stationary covariance models are used. This work introduces an adaptive tapering method in order to improve the performance of tapering in these problematic cases. This is achieved by introducing a computationally convenient class of compactly supported non-stationary covariance functions, combined with a new method for choosin…
Recently, a lot of effort has been paid to the efficient computation of Kriging predictors when observations are assimilated sequentially. In particular, Kriging update formulae enabling significant computational savings were derived in Barnes and Watson (1992), Gao et al. (1996), and Emery (2009). Taking advantage of the previous Kriging mean and variance calculations helps avoiding a costly $(n+1) \times (n+1)$ matrix inversion when adding one observation to the $n$ already available ones. In addition to traditional update formulae taking into account a single new observation, Emery (2009) a…
In the framework of the supervised learning of a real function defined on a space X , the so called Kriging method stands on a real Gaussian field defined on X. The Euclidean case is well known and has been widely studied. In this paper, we explore the less classical case where X is the non commutative finite group of permutations. In this setting, we propose and study an harmonic analysis of the covariance operators that enables to consider Gaussian processes models and forecasting issues. Our theory is motivated by statistical ranking problems.
In many environmental applications involving spatially-referenced data, limitations on the number and locations of observations motivate the need for practical and efficient models for spatial interpolation, or kriging. A key component of models for continuously-indexed spatial data is the covariance function, which is traditionally assumed to belong to a parametric class of stationary models. However, stationarity is rarely a realistic assumption. Alternative methods which more appropriately model the nonstationarity present in environmental processes often involve high-dimensional parameter …
This paper deals with the Gaussian process based approximation of a code which can be run at different levels of accuracy. This method, which is a particular case of co-kriging, allows us to improve a surrogate model of a complex computer code using fast approximations of it. In particular, we focus on the case of a large number of code levels on the one hand and on a Bayesian approach when we have two levels on the other hand. The main results of this paper are a new approach to estimate the model parameters which provides a closed form expression for an important parameter of the model (the …
Ecole des Mines de Saint Etienne（圣艾蒂安矿业学院）；CNRS（法国国家科学研究中心）
The Efficient Global Optimization (EGO) algorithm uses a conditional Gaus-sian Process (GP) to approximate an objective function known at a finite number of observation points and sequentially adds new points which maximize the Expected Improvement criterion according to the GP. The important factor that controls the efficiency of EGO is the GP covariance function (or kernel) which should be chosen according to the objective function. Traditionally, a pa-rameterized family of covariance functions is considered whose parameters are learned through statistical procedures such as maximum likeliho…
Pacific Northwest National Laboratory（太平洋西北国家实验室）；The University of Iowa（爱荷华大学）
In this work, we propose a framework that combines the approximation-theory-based multifidelity method and Gaussian-process-regression-based multifidelity method to achieve data-model convergence when stochastic simulation models and sparse accurate observation data are available. Specifically, the two types of multifidelity methods we use are the bifidelity and CoKriging methods. The new approach uses the bifidelity method to efficiently estimate the empirical mean and covariance of the stochastic simulation outputs, then it uses these statistics to construct a Gaussian process (GP) represent…
We present a multivariate Gaussian process regression approach for parameter field reconstruction based on the field's measurements collected at two different scales, the coarse and fine scales. The proposed approach treats the parameter field defined at fine and coarse scales as a bivariate Gaussian process with a parameterized multiscale covariance model. We employ a full bivariate Matérn kernel as multiscale covariance model, with shape and smoothness hyperparameters that account for the coarsening relation between fine and coarse fields. In contrast to similar multiscale kriging approaches…
Visual Spatial Learning: Single-Field Spatial Interpolation Using Convolutional Neural Networks
Centro de Matemática (CMAT), Universidade do Minho（数学中心（CMAT），明霍大学）；DEI-FEUP & INESC TEC, Universidade do Porto（FEUP-DEI与INESC TEC，波尔图大学）；Instituto Português do Mar e da Atmosfera, I. P. (IPMA, I. P.), Lisboa, Portugal（葡萄牙海洋与大气研究所（IPMA, I. P.），里斯本，葡萄牙）；Centro de Ciências do Mar e do Ambiente (MARE), Évora, Portugal（海洋与环境科学中心（MARE），埃维拉，葡萄牙）
This article focuses on the estimation and design aspects of a bivariate collocated cokriging experiment. For a large class of covariance matrices, a linear dependency criterion is identified, which allows the best linear unbiased estimator of the primary variable in a bivariate collocated cokriging setup to reduce to a univariate kriging estimator. Exact optimal designs for efficient prediction for such simple and ordinary reduced cokriging models with one-dimensional inputs are determined. Designs are found by minimizing the maximum and the integrated prediction variance, where the primary v…
We develop a framework for Gaussian processes regression constrained by boundary value problems. The framework may be applied to infer the solution of a well-posed boundary value problem with a known second-order differential operator and boundary conditions, but for which only scattered observations of the source term are available. Scattered observations of the solution may also be used in the regression. The framework combines co-kriging with the linear transformation of a Gaussian process together with the use of kernels given by spectral expansions in eigenfunctions of the boundary value …
Stochastic kriging has been widely employed for simulation metamodeling to predict the response surface of complex simulation models. However, its use is limited to cases where the design space is low-dimensional because, in general, the sample complexity (i.e., the number of design points required for stochastic kriging to produce an accurate prediction) grows exponentially in the dimensionality of the design space. The large sample size results in both a prohibitive sample cost for running the simulation model and a severe computational challenge due to the need to invert large covariance ma…
The paper covers the design and analysis of experiments to discriminate between two Gaussian process models, such as those widely used in computer experiments, kriging, sensor location and machine learning. Two frameworks are considered. First, we study sequential constructions, where successive design (observation) points are selected, either as additional points to an existing design or from the beginning of observation. The selection relies on the maximisation of the difference between the symmetric Kullback Leibler divergences for the two models, which depends on the observations, or on th…
Pontifical Catholic University of Rio de Janeiro（里约热内卢天主教大学）；University of York（约克大学）
In this work, we propose a subspace-based algorithm for DOA estimation which iteratively reduces the disturbance factors of the estimated data covariance matrix and incorporates prior knowledge which is gradually obtained on line. An analysis of the MSE of the reshaped data covariance matrix is carried out along with comparisons between computational complexities of the proposed and existing algorithms. Simulations focusing on closely-spaced sources, where they are uncorrelated and correlated, illustrate the improvements achieved.
Due to climate change and human activity, wildfires are expected to become more frequent and extreme worldwide, causing economic and ecological disasters. The deployment of preventive measures and operational forecasts can be aided by stochastic modeling that helps to understand and quantify the mechanisms governing the occurrence intensity. We here develop a point process framework for wildfire ignition points observed in the French Mediterranean basin since 1995, and we fit a spatio-temporal log-Gaussian Cox process with monthly temporal resolution in a Bayesian framework using the integrate…
Automated sensing instruments on satellites and aircraft have enabled the collection of massive amounts of high-resolution observations of spatial fields over large spatial regions. If these datasets can be efficiently exploited, they can provide new insights on a wide variety of issues. However, traditional spatial-statistical techniques such as kriging are not computationally feasible for big datasets. We propose a multi-resolution approximation (M-RA) of Gaussian processes observed at irregular locations in space. The M-RA process is specified as a linear combination of basis functions at m…
Pontifical Catholic University of Rio de Janeiro（里约热内卢天主教大学）；University of York（约克大学）
In this work, we present direction-of-arrival (DoA) estimation algorithms based on the Krylov subspace that effectively exploit prior knowledge of the signals that impinge on a sensor array. The proposed multi-step knowledge-aided iterative conjugate gradient (CG) (MS-KAI-CG) algorithms perform subtraction of the unwanted terms found in the estimated covariance matrix of the sensor data. Furthermore, we develop a version of MS-KAI-CG equipped with forward-backward averaging, called MS-KAI-CG-FB, which is appropriate for scenarios with correlated signals. Unlike current knowledge-aided methods,…
The log-Gaussian Cox process (LGCP) is a popular point process for modeling non-interacting spatial point patterns. This paper extends the LGCP model to handle data exhibiting fundamentally different behaviors in different subregions of the spatial domain. The aim of the analyst might be either to identify and classify these regions, to perform kriging, or to derive some properties of the parameters driving the random field in one or several of the subregions. The extension is based on replacing the latent Gaussian random field in the LGCP by a latent spatial mixture model. The mixture model i…
