Gaussian Processes (GPs) has experienced tremendous success in geoscience in general and for bio-geophysical parameter retrieval in the last years. GPs constitute a solid Bayesian framework to formulate many function approximation problems consistently. This paper reviews the main theoretical GP developments in the field. We review new algorithms that respect the signal and noise characteristics, that provide feature rankings automatically, and that allow applicability of associated uncertainty intervals to transport GP models in space and time. All these developments are illustrated in the field of geoscience and remote sensing at a local and global scales through a set of illustrative examples.
For many survey-based spatial modelling problems, responses are observed as spatially aggregated over survey regions due to limited resources. Covariates, from weather models and satellite imageries, can be observed at many different spatial resolutions, making the pre-processing of covariates a key challenge for any spatial modelling task. We propose a Gaussian process regression model to flexibly handle multiresolution covariates by employing an additive kernel that can efficiently aggregate features across resolutions. Compared to existing approaches that rely on resolution matching, our ap…
Universitat de València（瓦伦西亚大学）；University of Oxford（牛津大学）；German Aerospace Center（德国航空航天中心）；Max Planck Institute for Biogeochemistry（马克斯·普朗克生物地球化学研究所）
Earth observation (EO) by airborne and satellite remote sensing and in-situ observations play a fundamental role in monitoring our planet. In the last decade, machine learning and Gaussian processes (GPs) in particular has attained outstanding results in the estimation of bio-geo-physical variables from the acquired images at local and global scales in a time-resolved manner. GPs provide not only accurate estimates but also principled uncertainty estimates for the predictions, can easily accommodate multimodal data coming from different sensors and from multitemporal acquisitions, allow the in…
Gaussian process priors are commonly used in aerospace design for performing Bayesian optimization. Nonetheless, Gaussian processes suffer two significant drawbacks: outliers are a priori assumed unlikely, and the posterior variance conditioned on observed data depends only on the locations of those data, not the associated sample values. Student's-T processes are a generalization of Gaussian processes, founded on the Student's-T distribution instead of the Gaussian distribution. Student's-T processes maintain the primary advantages of Gaussian processes (kernel function, analytic update rule)…
We propose a simple method that combines neural networks and Gaussian processes. The proposed method can estimate the uncertainty of outputs and flexibly adjust target functions where training data exist, which are advantages of Gaussian processes. The proposed method can also achieve high generalization performance for unseen input configurations, which is an advantage of neural networks. With the proposed method, neural networks are used for the mean functions of Gaussian processes. We present a scalable stochastic inference procedure, where sparse Gaussian processes are inferred by stochast…
Aalto university（阿尔托大学）；Helsinki Institute for Information Technology HIIT（赫尔辛基信息技术研究所（HIIT））
We propose deep convolutional Gaussian processes, a deep Gaussian process architecture with convolutional structure. The model is a principled Bayesian framework for detecting hierarchical combinations of local features for image classification. We demonstrate greatly improved image classification performance compared to current Gaussian process approaches on the MNIST and CIFAR-10 datasets. In particular, we improve CIFAR-10 accuracy by over 10 percentage points.
Spatially-Heterogeneous Causal Bayesian Networks for Seismic Multi-Hazard Estimation: A Variational Approach with Gaussian Processes and Normalizing Flows
Johns Hopkins University（约翰斯·霍普金斯大学）；Data Science and AI Institute, Johns Hopkins University（约翰斯·霍普金斯大学数据科学与人工智能学院）
Gaussian processes are used in machine learning to learn input-output mappings from observed data. Gaussian process regression is based on imposing a Gaussian process prior on the unknown regressor function and statistically conditioning it on the observed data. In system identification, Gaussian processes are used to form time series prediction models such as non-linear finite-impulse response (NFIR) models as well as non-linear autoregressive (NARX) models. Gaussian process state-space models (GPSS) can be used to learn the dynamic and measurement models for a state-space representation of t…
Bayesian learning using Gaussian processes provides a foundational framework for making decisions in a manner that balances what is known with what could be learned by gathering data. In this dissertation, we develop techniques for broadening the applicability of Gaussian processes. This is done in two ways. Firstly, we develop pathwise conditioning techniques for Gaussian processes, which allow one to express posterior random functions as prior random functions plus a dependent update term. We introduce a wide class of efficient approximations built from this viewpoint, which can be randomly …
We provide the strong approximation of empirical copula processes by a Gaussian process. In addition we establish a strong approximation of the smoothed empirical copula processes and a law of iterated logarithm.
Université de Lille（里尔大学）；CNRS/CRIStAL（法国国家科学研究中心/CRIStAL实验室）；Inria Lille（法国里尔国家信息与自动化研究所）
Bayesian Optimization using Gaussian Processes is a popular approach to deal with the optimization of expensive black-box functions. However, because of the a priori on the stationarity of the covariance matrix of classic Gaussian Processes, this method may not be adapted for non-stationary functions involved in the optimization problem. To overcome this issue, a new Bayesian Optimization approach is proposed. It is based on Deep Gaussian Processes as surrogate models instead of classic Gaussian Processes. This modeling technique increases the power of representation to capture the non-station…
University of North Carolina at Charlotte（北卡罗来纳大学夏洛特分校）
Gaussian processes are one of the dominant approaches in Bayesian learning. Although the approach has been applied to numerous problems with great success, it has a few fundamental limitations. Multiple methods in literature have addressed these limitations. However, there has not been a comprehensive survey of the topics as of yet. Most existing surveys focus on only one particular variant of Gaussian processes and their derivatives. This survey details the core motivations for using Gaussian processes, their mathematical formulations, limitations, and research themes that have flourished ove…
Gaussian processes are ubiquitous in nature and engineering. A case in point is a class of neural networks in the infinite-width limit, whose priors correspond to Gaussian processes. Here we perturbatively extend this correspondence to finite-width neural networks, yielding non-Gaussian processes as priors. The methodology developed herein allows us to track the flow of preactivation distributions by progressively integrating out random variables from lower to higher layers, reminiscent of renormalization-group flow. We further develop a perturbative procedure to perform Bayesian inference wit…
As a generalization of the work in [Lee et al., 2017], this note briefly discusses when the prior of a neural network output follows a Gaussian process, and how a neural-network-induced Gaussian process is formulated. The posterior mean functions of such a Gaussian process regression lie in the reproducing kernel Hilbert space defined by the neural-network-induced kernel. In the case of two-layer neural networks, the induced Gaussian processes provide an interpretation of the reproducing kernel Hilbert spaces whose union forms a Barron space.
Australian Centre for Field Robotics, The University of Sydney（悉尼大学澳大利亚现场机器人中心）
This paper evaluates heterogeneous information fusion using multi-task Gaussian processes in the context of geological resource modeling. Specifically, it empirically demonstrates that information integration across heterogeneous information sources leads to superior estimates of all the quantities being modeled, compared to modeling them individually. Multi-task Gaussian processes provide a powerful approach for simultaneous modeling of multiple quantities of interest while taking correlations between these quantities into consideration. Experiments are performed on large scale real sensor da…
We propose a novel Bayesian nonparametric method to learn translation-invariant relationships on non-Euclidean domains. The resulting graph convolutional Gaussian processes can be applied to problems in machine learning for which the input observations are functions with domains on general graphs. The structure of these models allows for high dimensional inputs while retaining expressibility, as is the case with convolutional neural networks. We present applications of graph convolutional Gaussian processes to images and triangular meshes, demonstrating their versatility and effectiveness, com…
A gentle introduction to Gaussian processes (GPs). The three parts of the document consider GPs for regression, classification, and dimensionality reduction.
We investigate the Student-t process as an alternative to the Gaussian process as a nonparametric prior over functions. We derive closed form expressions for the marginal likelihood and predictive distribution of a Student-t process, by integrating away an inverse Wishart process prior over the covariance kernel of a Gaussian process model. We show surprising equivalences between different hierarchical Gaussian process models leading to Student-t processes, and derive a new sampling scheme for the inverse Wishart process, which helps elucidate these equivalences. Overall, we show that a Studen…
In this paper, we study central and non-central limit theorems for partial sum of functionals of general stationary Gaussian fields. We apply our result to study drift parameter estimation problems for some stochastic differential equations related to stationary Gaussian processes.
Real-world measurement noise in applications like robotics is often correlated in time, but we typically assume i.i.d. Gaussian noise for filtering. We propose general Gaussian Processes as a non-parametric model for correlated measurement noise that is flexible enough to accurately reflect correlation in time, yet simple enough to enable efficient computation. We show that this model accurately reflects the measurement noise resulting from vision-based Simultaneous Localization and Mapping (SLAM), and argue that it provides a flexible means of modeling measurement noise for a wide variety of …
The asymptotic analysis of covariance parameter estimation of Gaussian processes has been subject to intensive investigation. However, this asymptotic analysis is very scarce for non-Gaussian processes. In this paper, we study a class of non-Gaussian processes obtained by regular non-linear transformations of Gaussian processes. We provide the increasing-domain asymptotic properties of the (Gaussian) maximum likelihood and cross validation estimators of the covariance parameters of a non-Gaussian process of this class. We show that these estimators are consistent and asymptotically normal, alt…
Infrastructure recovery time estimation is critical to disaster management and planning. Inspired by recent resilience planning initiatives, we consider a situation where experts are asked to estimate the time for different infrastructure systems to recover to certain functionality levels after a scenario hazard event. We propose a methodological framework to use expert-elicited data to estimate the expected recovery time curve of a particular infrastructure system. This framework uses the Gaussian process regression (GPR) to capture the experts' estimation-uncertainty and satisfy known physic…
Deep Learning Gaussian Processes (DL-GP) are proposed as a methodology for analyzing (approximating) computer models that produce heteroskedastic and high-dimensional output. Computer simulation models have many areas of applications, including social-economic processes, agriculture, environmental, biology, engineering and physics problems. A deterministic transformation of inputs is performed by deep learning and predictions are calculated by traditional Gaussian Processes. We illustrate our methodology using a simulation of motorcycle accidents and simulations of an Ebola outbreak. Finally, …
There are all kinds of weak dependence. For example, strong mixing. Short-range dependence (SRD) is also a form of weak dependence. It occurs in the context of processes that are subordinated to the Gaussian. Is a SRD process strong mixing if the underlying Gaussian process is long-range dependent? We show that this is not necessarily the case.
Photometric variability of a directly imaged exo-Earth conveys spatial information on its surface and can be used to retrieve a two-dimensional geography and axial tilt of the planet (spin-orbit tomography). In this study, we relax the assumption of the static geography and present a computationally tractable framework for dynamic spin-orbit tomography applicable to the time-varying geography. First, a Bayesian framework of static spin-orbit tomography is revisited using analytic expressions of the Bayesian inverse problem with a Gaussian prior. We then extend this analytic framework to a time…
In this paper we derive Piterbarg's max-discretisation theorem for two different grids considering centered stationary vector Gaussian processes. So far in the literature results in this direction have been derived for the joint distribution of the maximum of Gaussian processes over $[0,T]$ and over a grid $ \mathfrak{R}(δ_1(T))=\{kδ_1(T): k=0,1,\cdots\}$. In this paper we extend recent findings by considering additionally the \bE{maximum} over another grid $ \mathfrak{R}(δ_2(T))$. We derive the joint limiting distribution of maximum of stationary Gaussian vector processes for different choice…
We present a non-parametric joint estimation method for fMRI task activation values and the hemodynamic response function (HRF). The HRF is modeled as a Gaussian process, making continuous evaluation possible for jittered paradigms and providing a variance estimate at each point.
Various approaches to stochastic processes exist, noting that key properties such as measurability and continuity are not trivially satisfied. We introduce a new theory for Gaussian processes using improper linear functionals. Using a collection of i.i.d. standard normal variables, we define Gaussian white noise and discuss its properties. This is extended to general Gaussian processes on Hilbert space, where the variance is allowed to be any suitable operator. Our main focus is $L^2$ spaces, and we discuss criteria for Gaussian processes to be continuous in this setting. Finally, we outline a…
Gaussian processes occupy one of the leading places in modern statistics and probability theory due to their importance and a wealth of strong results. The common use of Gaussian processes is in connection with problems related to estimation, detection, and many statistical or machine learning models. With the fast development of Gaussian process applications, it is necessary to consolidate the fundamentals of vector-valued stochastic processes, in particular multivariate Gaussian processes, which is the essential theory for many applied problems with multiple correlated responses. In this pap…
In this paper, we study the quantitative behavior of a spatial risk measure corresponding to a damage function and a region, taking into account the spatial dependence of the underlying process. This kind of risk measure has already been introduced and studied for some max-stable processes in [Koch2015]. In this paper, we consider isotropic Gaussian processes and the excess damage function over a threshold. We performed a simulation study and a real data study.
In this paper, we have provided an overview of cosmic ray effects on terrestrial processes such as electrical properties, global electric circuit, lightning, cloud formation, cloud coverage, atmospheric temperature, space weather phenomena, climate, etc. It is suggested that cosmic rays control short term and long term variation in climate. There are many basic phenomena which need further study and require new and long term data set. Some of these have been pointed out.
The detrended fluctuation analysis (DFA) is extensively useful in stochastic processes to unveil the long-term correlation. Here, we apply the DFA to point processes that mimick earthquake data. The point processes are synthesized by a model similar to the Epidemic-Type Aftershock Sequence model, and we apply the DFA to time series $N(t)$ of the point processes, where $N(t)$ is the cumulative number of events up to time $t$. Crossover phenomena are found in the DFA for these time series, and extensive numerical simulations suggest that the crossover phenomena are signatures of non-stationarity…
In meteorology, engineering and computer sciences, data assimilation is routinely employed as the optimal way to combine noisy observations with prior model information for obtaining better estimates of a state, and thus better forecasts, than can be achieved by ignoring data uncertainties. Earthquake forecasting, too, suffers from measurement errors and partial model information and may thus gain significantly from data assimilation. We present perhaps the first fully implementable data assimilation method for earthquake forecasts generated by a point-process model of seismicity. We test the …
Geometrical properties of landscapes result from the geological processes that have acted through time. The quantitative analysis of natural relief represents an objective form of aiding in the visual interpretation of landscapes, as studies on coastlines, river networks, and global topography, have shown. Still, an open question is whether a clear relationship between the quantitative properties of landscapes and the dominant geomorphologic processes that originate them can be established. In this contribution, we show that the geometry of topographic isolines is an appropriate observable to …
In a recent article by Dickey, J. O., Marcus, S.L. and O. de Viron, 2011, the authors show evidences for correlations in the multi-decadal variability of Earth's rotation rate [i.e., length of day (LOD)], the angular momentum of the core (CAM), and natural surface air temperature (SAT). Previous investigators have already found that the LOD fluctuations are largely attributed to core-mantle interactions and that the SAT appears strongly anti-correlated with the decadal LOD. As the above authors note, the cause of this common variability needs to be further investigated and studied. In fact, "s…
The external galactic key factor for developing of massive geochronological boundaries, as well as mass extinctions (ME) is proved based on chronological relationships of the impulses of globally short processes. External galactic key factor is also proved based on the coincidence of the scales of their developing processes, their regular periodicity in the formation of the boundaries of the Phanerozoic epochs and the presence of a unifying complex of cause-effect relationships for boundary processes. The hypotheses of the impact origin of ocean basins and plumes are substantiated, the assumpt…
Investigating the polarity time scales for the last 118 My a Bernoulli process with p=0.5 overriding with a Gaussian noise have been detected for the R-N magnetic reversals. The Bernoulli trials are separated by 282 ky. The frequency of the Bernoulli process correlates to planetary cycles and consistent with an impact triggered global cooling mechanism. The detected pattern allows calculating the probability of an upcoming R-N reversal between any time intervals.
