The stochastic partial differential equation (SPDE) approach is widely used for modeling large spatial datasets. It is based on representing a Gaussian random field $u$ on $\mathbb{R}^d$ as the solution of an elliptic SPDE $L^βu = \mathcal{W}$ where $L$ is a second-order differential operator, $2β$ (belongs to natural number starting from 1) is a positive parameter that controls the smoothness of $u$ and $\mathcal{W}$ is Gaussian white noise. A few approaches have been suggested in the literature to extend the approach to allow for any smoothness parameter satisfying $β>d/4$. Even though those approaches work well for simulating SPDEs with general smoothness, they are less suitable for Bayesian inference since they do not provide approximations which are Gaussian Markov random fields (GMRFs) as in the original SPDE approach. We address this issue by proposing a new method based on approximating the covariance operator $L^{-2β}$ of the Gaussian field $u$ by a finite element method combined with a rational approximation of the fractional power. This results in a numerically stable GMRF approximation which can be combined with the integrated nested Laplace approximation (INLA) method for fast Bayesian inference. A rigorous convergence analysis of the method is performed and the accuracy of the method is investigated with simulated data. Finally, we illustrate the approach and corresponding implementation in the R package rSPDE via an application to precipitation data which is analyzed by combining the rSPDE package with the R-INLA software for full Bayesian inference.
Gaussian random fields (GRFs) constitute an important part of spatial modelling, but can be computationally infeasible for general covariance structures. An efficient approach is to specify GRFs via stochastic partial differential equations (SPDEs) and derive Gaussian Markov random field (GMRF) approximations of the solutions. We consider the construction of a class of non-stationary GRFs with varying local anisotropy, where the local anisotropy is introduced by allowing the coefficients in the SPDE to vary with position. This is done by using a form of diffusion equation driven by Gaussian wh…
In this paper we propose a new approach for constructing \emph{multivariate} Gaussian random fields (GRFs) with oscillating covariance functions through systems of stochastic partial differential equations (SPDEs). We discuss how to build systems of SPDEs that introduces oscillation characteristics in the covariance functions of the multivariate GRFs. By choosing different parametrization of the equations, some GRFs can be made with oscillating covariance functions but other fields can have Matérn covariance functions or close to Matérn covariance functions. The multivariate GRFs constructed b…
Gaussian processes and random fields have a long history, covering multiple approaches to representing spatial and spatio-temporal dependence structures, such as covariance functions, spectral representations, reproducing kernel Hilbert spaces, and graph based models. This article describes how the stochastic partial differential equation approach to generalising Matérn covariance models via Hilbert space projections connects with several of these approaches, with each connection being useful in different situations. In addition to an overview of the main ideas, some important extensions, theo…
The article studies non-Gaussian extensions of a recently discovered link between certain Gaussian random fields, expressed as solutions to stochastic partial differential equations (SPDEs), and Gaussian Markov random fields. The focus is on non-Gaussian random fields with Matérn covariance functions, and in particular we show how the SPDE formulation of a Laplace moving average model can be used to obtain an efficient simulation method as well as an accurate parameter estimation technique for the model. This should be seen as a demonstration of how these techniques can be used, and generaliza…
A popular approach for modeling and inference in spatial statistics is to represent Gaussian random fields as solutions to stochastic partial differential equations (SPDEs) of the form $L^βu = \mathcal{W}$, where $\mathcal{W}$ is Gaussian white noise, $L$ is a second-order differential operator, and $β>0$ is a parameter that determines the smoothness of $u$. However, this approach has been limited to the case $2β\in\mathbb{N}$, which excludes several important models and makes it necessary to keep $β$ fixed during inference. We propose a new method, the rational SPDE approach, which in spatial…
In this paper a new approach for constructing \emph{multivariate} Gaussian random fields (GRFs) using systems of stochastic partial differential equations (SPDEs) has been introduced and applied to simulated data and real data. By solving a system of SPDEs, we can construct multivariate GRFs. On the theoretical side, the notorious requirement of non-negative definiteness for the covariance matrix of the GRF is satisfied since the constructed covariance matrices with this approach are automatically symmetric positive definite. Using the approximate stochastic weak solutions to the systems of SP…
We present a new technique, based on semivariogram methodology, for obtaining point estimates for use in prior modeling for solving Bayesian inverse problems. This method requires a connection between Gaussian processes with covariance operators defined by the Matérn covariance function and Gaussian processes with precision (inverse-covariance) operators defined by the Green's functions of a class of elliptic stochastic partial differential equations (SPDEs). We present a detailed mathematical description of this connection. We will show that there is an equivalence between these two Gaussian …
A non-stationary spatial Gaussian random field (GRF) is described as the solution of an inhomogeneous stochastic partial differential equation (SPDE), where the covariance structure of the GRF is controlled by the coefficients in the SPDE. This allows for a flexible way to vary the covariance structure, where intuition about the resulting structure can be gained from the local behaviour of the differential equation. Additionally, computations can be done with computationally convenient Gaussian Markov random fields which approximate the true GRFs. The model is applied to a dataset of annual pr…
We present a novel approach for the analysis of multivariate case-control georeferenced data using Bayesian inference in the context of disease mapping, where the spatial distribution of different types of cancers is analyzed. Extending other methodology in point pattern analysis, we propose a log-Gaussian Cox process for point pattern of cases and the controls, which accounts for risk factors, such as exposure to pollution sources, and includes a term to measure spatial residual variation. For each disease, its intensity is modeled on a baseline spatial effect (estimated from both controls an…
This paper is an extension of Parts I and Ia of a series about Nu-class generalized functions. We provide hand-generated algebraic expressions for integrals of single Matern-covariance functions, as well as for products of two Matern-covariance functions, for all odd-half-integer class parameters. These are useful both for IMSPE-optimal design software and for testing universality of Nu-class generalized-function properties, across covariance classes.
We propose a determinant-free approach for simulation-based Bayesian inference in high-dimensional Gaussian models. We introduce auxiliary variables with covariance equal to the inverse covariance of the model. The joint probability of the auxiliary model can be computed without evaluating determinants, which are often hard to compute in high dimensions. We develop a Markov chain Monte Carlo sampling scheme for the auxiliary model that requires no more than the application of inverse-matrix-square-roots and the solution of linear systems. These operations can be performed at large scales with …
Early approaches to multiple-output Gaussian processes (MOGPs) relied on linear combinations of independent, latent, single-output Gaussian processes (GPs). This resulted in cross-covariance functions with limited parametric interpretation, thus conflicting with the ability of single-output GPs to understand lengthscales, frequencies and magnitudes to name a few. On the contrary, current approaches to MOGP are able to better interpret the relationship between different channels by directly modelling the cross-covariances as a spectral mixture kernel with a phase shift. We extend this rationale…
The partial stochastic realization of periodic processes from finite covariance data has recently been solved by Lindquist and Picci based on convex optimization of a generalized entropy functional. The meaning and the role of this criterion have an unclear origin. In this paper we propose a solution based on a nonlinear generalization of the classical Yule-Walker type equations and on a new iterative algorithm which is shown to converge to the same (unique) solution of the variational problem. This provides a conceptual link to the variational principles and at the same time yields a robust a…
We consider estimation of the parameters of a Gaussian Stochastic Process (GaSP), in the context of emulation (approximation) of computer models for which the outcomes are real-valued scalars. The main focus is on estimation of the GaSP parameters through various generalized maximum likelihood methods, mostly involving finding posterior modes; this is because full Bayesian analysis in computer model emulation is typically prohibitively expensive. The posterior modes that are studied arise from objective priors, such as the reference prior. These priors have been studied in the literature for t…
Gaussian processes are powerful, yet analytically tractable models for supervised learning. A Gaussian process is characterized by a mean function and a covariance function (kernel), which are determined by a model selection criterion. The functions to be compared do not just differ in their parametrization but in their fundamental structure. It is often not clear which function structure to choose, for instance to decide between a squared exponential and a rational quadratic kernel. Based on the principle of approximation set coding, we develop a framework for model selection to rank kernels …
Isotropic covariance structures can be unreasonable for phenomena in three-dimensional spaces such as the ocean. In the ocean, the variability of the response may vary with depth, and ocean currents may lead to spatially varying anisotropy. We construct a class of non-stationary anisotropic Gaussian random fields (GRFs) in three dimensions through stochastic partial differential equations (SPDEs) where computations are done using Gaussian Markov random field approximations. The approach is proven in a simulation study where the amount of data required to estimate these models is explored. Then…
The ability to quantify distinctness of a cluster structure is fundamental for certain simulation studies, in particular for those comparing performance of different classification algorithms. The intrinsic integral measure based on the overlap of corresponding mixture components is often analytically intractable. This is also the case for Gaussian mixture models with unequal covariance matrices when space dimension $d > 1$. In this work we focus on Gaussian mixture models and at the sample level we assume the class assignments to be known. We derive a measure of component overlap based on eig…
Columbia University（哥伦比亚大学）；The Broad Institute（博德研究所）；Facebook（脸书）；Cornell University（康奈尔大学）；University of Pennsylvania（宾夕法尼亚大学）；Uber AI（优步人工智能）
Matrix square roots and their inverses arise frequently in machine learning, e.g., when sampling from high-dimensional Gaussians $\mathcal{N}(\mathbf 0, \mathbf K)$ or whitening a vector $\mathbf b$ against covariance matrix $\mathbf K$. While existing methods typically require $O(N^3)$ computation, we introduce a highly-efficient quadratic-time algorithm for computing $\mathbf K^{1/2} \mathbf b$, $\mathbf K^{-1/2} \mathbf b$, and their derivatives through matrix-vector multiplication (MVMs). Our method combines Krylov subspace methods with a rational approximation and typically achieves $4$ d…
Generating large-scale samples of stationary random fields is of great importance in the fields such as geomaterial modeling and uncertainty quantification. Traditional methodologies based on covariance matrix decomposition have the diffculty of being computationally expensive, which is even more serious when the dimension of the random field is large. This paper proposes an effcient stochastic realization approach for sampling Gaussian stationary random fields from a systems and control point of view. Specifically, we take the exponential and Gaussian covariance functions as examples and make…
