Statistical analysis of max-stable processes used to model spatial extremes has been limited by the difficulty in calculating the joint likelihood function. This precludes all standard likelihood-based approaches, including Bayesian approaches. In this paper we present a Bayesian approach through the use of approximate Bayesian computing. This circumvents the need for a joint likelihood function by instead relying on simulations from the (unavailable) likelihood. This method is compared with an alternative approach based on the composite likelihood. We demonstrate that approximate Bayesian computing can result in a lower mean square error than the composite likelihood approach when estimating the spatial dependence of extremes, though at an appreciably higher computational cost. We also illustrate the performance of the method with an application to US temperature data to estimate the risk of crop loss due to an unlikely freeze event.
Statistical extreme value theory is concerned with the use of asymptotically motivated models to describe the extreme values of a process. A number of commonly used models are valid for observed data that exceed some high threshold. However, in practice a suitable threshold is unknown and must be determined for each analysis. While there are many threshold selection methods for univariate extremes, there are relatively few that can be applied in the multivariate setting. In addition, there are only a few Bayesian-based methods, which are naturally attractive in the modelling of extremes due to…
This paper provides a review of Approximate Bayesian Computation (ABC) methods for carrying out Bayesian posterior inference, through the lens of density estimation. We describe several recent algorithms and make connection with traditional approaches. We show advantages and limitations of models based on parametric approaches and we then draw attention to developments in machine learning, which we believe have the potential to make ABC scalable to higher dimensions and may be the future direction for research in this area.
Approximate Bayesian computation (ABC) methods, which are applicable when the likelihood is difficult or impossible to calculate, are an active topic of current research. Most current ABC algorithms directly approximate the posterior distribution, but an alternative, less common strategy is to approximate the likelihood function. This has several advantages. First, in some problems, it is easier to approximate the likelihood than to approximate the posterior. Second, an approximation to the likelihood allows reference analyses to be constructed based solely on the likelihood. Third, it is stra…
We describe a simple method for making inference on a functional of a multivariate distribution. The method is based on a copula representation of the multivariate distribution and it is based on the properties of an Approximate Bayesian Monte Carlo algorithm, where the proposed values of the functional of interest are weighed in terms of their empirical likelihood. This method is particularly useful when the "true" likelihood function associated with the working model is too costly to evaluate or when the working model is only partially specified.
Bayesian inference for functional extreme events defined via partially unobserved processes
Institute for Stochastics and Applications, University of Stuttgart（斯图加特大学随机与应用研究所）；Stuttgart Center for Simulation Science (SC SimTech), University of Stuttgart（斯图加特大学斯图加特仿真科学中心）
This note discusses Watson and Holmes (2016) and their pro- posals towards more robust Bayesian decisions. While we acknowledge and commend the authors for setting new and all-encompassing prin- ciples of Bayesian robustness, and we appreciate the strong anchoring of those within a decision-theoretic referential, we remain uncertain as to which extent such principles can be applied outside binary de- cisions. We also wonder at the ultimate relevance of Kullback-Leibler neighbourhoods to characterise robustness and favour extensions along non-parametric axes.
We combine conditioning techniques with sparse grid quadrature rules to develop a computationally efficient method to approximate marginal, but not necessarily univariate, posterior quantities, yielding approximate Bayesian inference via Sparse grid Quadrature Evaluation (BISQuE) for hierarchical models. BISQuE reformulates posterior quantities as weighted integrals of conditional quantities, such as densities and expectations. Sparse grid quadrature rules allow computationally efficient approximation of high dimensional integrals, which appear in hierarchical models with many hyperparameters.…
We propose a Bayesian hierarchical model for spatial extremes on a large domain. In the data layer a Gaussian elliptical copula having generalized extreme value (GEV) marginals is applied. Spatial dependence in the GEV parameters are captured with a latent spatial regression with spatially varying coefficients. Using a composite likelihood approach, we are able to efficiently incorporate a large precipitation dataset, which includes stations with missing data. The model is demonstrated by application to fall precipitation extremes at approximately 2600 stations covering the western United Stat…
Estimation of finite mixture models when the mixing distribution support is unknown is an important problem. This paper gives a new approach based on a marginal likelihood for the unknown support. Motivated by a Bayesian Dirichlet prior model, a computationally efficient stochastic approximation version of the marginal likelihood is proposed and large-sample theory is presented. By restricting the support to a finite grid, a simulated annealing method is employed to maximize the marginal likelihood and estimate the support. Real and simulated data examples show that this novel stochastic appro…
Motivated by the analysis of extreme rainfall data, we introduce a general Bayesian hierarchical model for estimating the probability distribution of extreme values of intermittent random sequences, a common problem in geophysical and environmental science settings. The approach presented here relaxes the asymptotic assumption typical of the traditional extreme value (EV) theory, and accounts for the possible underlying variability in the distribution of event magnitudes and occurrences, which are described through a latent temporal process. Focusing on daily rainfall extremes, the structure o…
The max-stable process is an asymptotically justified model for spatial extremes. In particular, we focus on the hierarchical extreme-value process (HEVP), which is a particular max-stable process that is conducive to Bayesian computing. The HEVP and all max-stable process models are parametric and impose strong assumptions including that all marginal distributions belong to the generalized extreme value family and that nearby sites are asymptotically dependent. We generalize the HEVP by relaxing these assumptions to provide a wider class of marginal distributions via a Dirichlet process prior…
The Brown-Resnick max-stable process has proven to be well-suited for modeling extremes of complex environmental processes, but in many applications its likelihood function is intractable and inference must be based on a composite likelihood, thereby preventing the use of classical Bayesian techniques. In this paper we exploit a case in which the full likelihood of a Brown-Resnick process can be calculated, using componentwise maxima and their partitions in terms of individual events, and we propose two new approaches to inference. The first estimates the partitions using declustering, while t…
Both marginal and dependence features must be described when modelling the extremes of a stationary time series. There are standard approaches to marginal modelling, but long- and short-range dependence of extremes may both appear. In applications, an assumption of long-range independence often seems reasonable, but short-range dependence, i.e., the clustering of extremes, needs attention. The extremal index $0<θ\le 1$ is a natural limiting measure of clustering, but for wide classes of dependent processes, including all stationary Gaussian processes, it cannot distinguish dependent processes …
In this study, we examine a Bayesian approach to analyze extreme daily rainfall amounts and forecast return-levels. Estimating the probability of occurrence and quantiles of future extreme events is important in many applications, including civil engineering and the design of public infrastructure. In contrast to traditional analysis, which use point estimates to accomplish this goal, the Bayesian method utilizes the complete posterior density derived from the observations. The Bayesian approach offers the benefit of well defined credible (confidence) intervals, improved forecasting, and the a…
Describing the complex dependence structure of extreme phenomena is particularly challenging. To tackle this issue we develop a novel statistical algorithm that describes extremal dependence taking advantage of the inherent hierarchical dependence structure of the max-stable nested logistic distribution and that identifies possible clusters of extreme variables using reversible jump Markov chain Monte Carlo techniques. Parsimonious representations are achieved when clusters of extreme variables are found to be completely independent. Moreover, we significantly decrease the computational comple…
Complex simulators have become a ubiquitous tool in many scientific disciplines, providing high-fidelity, implicit probabilistic models of natural and social phenomena. Unfortunately, they typically lack the tractability required for conventional statistical analysis. Approximate Bayesian computation (ABC) has emerged as a key method in simulation-based inference, wherein the true model likelihood and posterior are approximated using samples from the simulator. In this paper, we draw connections between ABC and generalized Bayesian inference (GBI). First, we re-interpret the accept/reject step…
Composite likelihoods are increasingly used in applications where the full likelihood is analytically unknown or computationally prohibitive. Although the maximum composite likelihood estimator has frequentist properties akin to those of the usual maximum likelihood estimator, Bayesian inference based on composite likelihoods has yet to be explored. In this paper we investigate the use of the Metropolis--Hastings algorithm to compute a pseudo-posterior distribution based on the composite likelihood. Two methodologies for adjusting the algorithm are presented and their performance on approximat…
The Bayesian lasso is well-known as a Bayesian alternative for Lasso. Although the advantage of the Bayesian lasso is capable of full probabilistic uncertain quantification for parameters, the corresponding posterior distribution can be sensitive to outliers. To overcome such problem, robust Bayesian regression models have been proposed in recent years. In this paper, we consider the robust and efficient estimation for the Bayesian Huberized lasso regression in fully Bayesian perspective. A new posterior computation algorithm for the Bayesian Huberized lasso regression is proposed. The propose…
Bayesian Mixture Models for Heterogeneous Extremes
Unité Biostatistique et Processus Spatiaux, INRAE, France（生物统计与空间过程单元，法国国家农业食品与环境研究院）；School of Mathematics, University of Edinburgh, UK（爱丁堡大学数学学院）；Department of Mathematics, University of Aveiro, Portugal（阿威罗大学数学系）；Department of Environmental Sciences, Informatics and Statistics, Ca’Foscari University of Venice, Italy（威尼斯卡福斯卡里大学环境科学、信息学与统计学系）
Bayesian synthetic likelihood (BSL) is now an established method for conducting approximate Bayesian inference in models where, due to the intractability of the likelihood function, exact Bayesian approaches are either infeasible or computationally too demanding. Implicit in the application of BSL is the assumption that the data generating process (DGP) can produce simulated summary statistics that capture the behaviour of the observed summary statistics. We demonstrate that if this compatibility between the actual and assumed DGP is not satisfied, i.e., if the model is misspecified, BSL can y…
Approximate Bayesian computation (ABC) is a method for Bayesian inference when the likelihood is unavailable but simulating from the model is possible. However, many ABC algorithms require a large number of simulations, which can be costly. To reduce the computational cost, Bayesian optimisation (BO) and surrogate models such as Gaussian processes have been proposed. Bayesian optimisation enables one to intelligently decide where to evaluate the model next but common BO strategies are not designed for the goal of estimating the posterior distribution. Our paper addresses this gap in the litera…
In this paper we propose a model with a Dirichlet process mixture of gamma densities in the bulk part below threshold and a generalized Pareto density in the tail for extreme value estimation. The proposed model is simple and flexible allowing us posterior density estimation and posterior inference for high quantiles. The model works well even for small sample sizes and in the absence of prior information. We evaluate the performance of the proposed model through a simulation study. Finally, the proposed model is applied to a real environmental data.
The aghq package for implementing approximate Bayesian inference using adaptive quadrature is introduced. The method and software are described, and use of the package in making approximate Bayesian inferences in several challenging low- and high-dimensional models is illustrated. Examples include an infectious disease model; an astrostatistical model for estimating the mass of the Milky Way; two examples in non-Gaussian model-based geostatistics including one incorporating zero-inflation which is not easily fit using other methods; and a model for zero-inflated, overdispersed count data. The …
Approximate Bayesian Computation (ABC) methods are used to approximate posterior distributions in models with unknown or computationally intractable likelihoods. Both the accuracy and computational efficiency of ABC depend on the choice of summary statistic, but outside of special cases where the optimal summary statistics are known, it is unclear which guiding principles can be used to construct effective summary statistics. In this paper we explore the possibility of automating the process of constructing summary statistics by training deep neural networks to predict the parameters from arti…
The Generalized Pareto (GP) and Generalized extreme value (GEV) distributions play an important role in extreme value analyses, as models for threshold excesses and block maxima respectively. For each of these distributions we consider Bayesian inference using "reference" prior distributions (in the general sense of priors constructed using formal rules) for the model parameters, specifically a Jeffreys prior, the maximal data information (MDI) prior and independent uniform priors on separate model parameters. We investigate the important issue of whether these improper priors lead to proper p…
In this paper we show that there is a link between approximate Bayesian methods and prior robustness. We show that what is typically recognized as an approximation to the likelihood, either due to the simulated data as in the Approximate Bayesian Computation (ABC) methods or due to the functional approximation to the likelihood, can instead also be viewed upon as an implicit exercise in prior robustness. We first define two new classes of priors for the cases where the sufficient statistics is available, establish their mathematical properties and show, for a simple illustrative example, that …
We propose a novel Bayesian nonparametric method for hierarchical modelling on a set of related density functions, where grouped data in the form of samples from each density function are available. Borrowing strength across the groups is a major challenge in this context. To address this problem, we introduce a hierarchically structured prior, defined over a set of univariate density functions, using convenient transformations of Gaussian processes. Inference is performed through approximate Bayesian computation (ABC), via a novel functional regression adjustment. The performance of the propo…
Also known as likelihood-free methods, approximate Bayesian computational (ABC) methods have appeared in the past ten years as the most satisfactory approach to untractable likelihood problems, first in genetics then in a broader spectrum of applications. However, these methods suffer to some degree from calibration difficulties that make them rather volatile in their implementation and thus render them suspicious to the users of more traditional Monte Carlo methods. In this survey, we study the various improvements and extensions made to the original ABC algorithm over the recent years.
For extreme value estimation we propose to use a model with a Dirichlet process mixture of gamma densities in the center and generalized Pareto densities for the tails. Due to the randomness in the center and a heavy tailed density in the tails density estimation and posterior inference for high quantiles are possible. The approach can be used in a "default" manner on the positive reals because it works when prior information is unavailable. The proposed model can be easy to implement and a sensitivity analysis is provided. We applied the proposed model for simulated and real data sets.
Rich data generating mechanisms are ubiquitous in this age of information and require complex statistical models to draw meaningful inference. While Bayesian analysis has seen enormous development in the last 30 years, benefitting from the impetus given by the successful application of Markov chain Monte Carlo (MCMC) sampling, the combination of big data and complex models conspire to produce significant challenges for the traditional MCMC algorithms. We review modern algorithmic developments addressing the latter and compare their performance using numerical experiments.
Approximate Bayesian Computation (ABC) is a popular method for approximate inference in generative models with intractable but easy-to-sample likelihood. It constructs an approximate posterior distribution by finding parameters for which the simulated data are close to the observations in terms of summary statistics. These statistics are defined beforehand and might induce a loss of information, which has been shown to deteriorate the quality of the approximation. To overcome this problem, Wasserstein-ABC has been recently proposed, and compares the datasets via the Wasserstein distance betwee…
Dalian University of Technology（大连理工大学）；University College London（伦敦大学学院）
We present a Bayesian scheme for the approximate diagonalisation of several square matrices which are not necessarily symmetric. A Gibbs sampler is derived to simulate samples of the common eigenvectors and the eigenvalues for these matrices. Several synthetic examples are used to illustrate the performance of the proposed Gibbs sampler and we then provide comparisons to several other joint diagonalization algorithms, which shows that the Gibbs sampler achieves the state-of-the-art performance on the examples considered. As a byproduct, the output of the Gibbs sampler could be used to estimate…
Approximate Bayesian computation (ABC) methods perform inference on model-specific parameters of mechanistically motivated parametric statistical models when evaluating likelihoods is difficult. Central to the success of ABC methods is computationally inexpensive simulation of data sets from the parametric model of interest. However, when simulating data sets from a model is so computationally expensive that the posterior distribution of parameters cannot be adequately sampled by ABC, inference is not straightforward. We present approximate approximate Bayesian computation" (AABC), a class of …
We discuss an approach for deriving robust posterior distributions from $M$-estimating functions using Approximate Bayesian Computation (ABC) methods. In particular, we use $M$-estimating functions to construct suitable summary statistics in ABC algorithms. The theoretical properties of the robust posterior distributions are discussed. Special attention is given to the application of the method to linear mixed models. Simulation results and an application to a clinical study demonstrate the usefulness of the method. An R implementation is also provided in the robustBLME package.
Model-assisted estimation with complex survey data is an important practical problem in survey sampling. When there are many auxiliary variables, selecting significant variables associated with the study variable would be necessary to achieve efficient estimation of population parameters of interest. In this paper, we formulate a regularized regression estimator in the framework of Bayesian inference using the penalty function as the shrinkage prior for model selection. The proposed Bayesian approach enables us to get not only efficient point estimates but also reasonable credible intervals. R…
Current literature on posterior approximation for Bayesian inference offers many alternative methods. Does our chosen approximation scheme work well on the observed data? The best existing generic diagnostic tools treating this kind of question by looking at performance averaged over data space, or otherwise lack diagnostic detail. However, if the approximation is bad for most data, but good at the observed data, then we may discard a useful approximation. We give graphical diagnostics for posterior approximation at the observed data. We estimate a "distortion map" that acts on univariate marg…
Large wildfires pose a major environmental concern, and precise maps of fire risk can improve disaster relief planning. Fosberg Fire Weather Index (FFWI) is often used to measure wildfire risk; FFWI exhibits non-Gaussian marginal distributions as well as strong spatiotemporal extremal dependence and thus, modeling FFWI using geostatistical models like Gaussian processes is questionable. Extreme value theory (EVT)-driven models like max-stable processes are theoretically appealing but are computationally demanding and applicable only for threshold exceedances or block maxima. Disaster managemen…
Both Approximate Bayesian Computation (ABC) and composite likelihood methods are useful for Bayesian and frequentist inference, respectively, when the likelihood function is intractable. We propose to use composite likelihood score functions as summary statistics in ABC in order to obtain accurate approximations to the posterior distribution. This is motivated by the use of the score function of the full likelihood, and extended to general unbiased estimating functions in complex models. Moreover, we show that if the composite score is suitably standardised, the resulting ABC procedure is inva…
Approximate Bayesian computation allows for statistical analysis in models with intractable likelihoods. In this paper we consider the asymptotic behaviour of the posterior distribution obtained by this method. We give general results on the rate at which the posterior distribution concentrates on sets containing the true parameter, its limiting shape, and the asymptotic distribution of the posterior mean. These results hold under given rates for the tolerance used within the method, mild regularity conditions on the summary statistics, and a condition linked to identification of the true para…
This paper proposes approaches for the analysis of multiple changepoint models when dependency in the data is modelled through a hierarchical Gaussian Markov random field. Integrated nested Laplace approximations are used to approximate data quantities, and an approximate filtering recursions approach is proposed for savings in compuational cost when detecting changepoints. All of these methods are simulation free. Analysis of real data demonstrates the usefulness of the approach in general. The new models which allow for data dependence are compared with conventional models where data within …
We provide the first stochastic convergence rates for a family of adaptive quadrature rules used to normalize the posterior distribution in Bayesian models. Our results apply to the uniform relative error in the approximate posterior density, the coverage probabilities of approximate credible sets, and approximate moments and quantiles, therefore guaranteeing fast asymptotic convergence of approximate summary statistics used in practice. The family of quadrature rules includes adaptive Gauss-Hermite quadrature, and we apply this rule in two challenging low-dimensional examples. Further, we dem…
Approximate Bayesian Computation (ABC) has become increasingly prominent as a method for conducting parameter inference in a range of challenging statistical problems, most notably those characterized by an intractable likelihood function. In this paper, we focus on the use of ABC not as a tool for parametric inference, but as a means of generating probabilistic forecasts; or for conducting what we refer to as `approximate Bayesian forecasting'. The four key issues explored are: i) the link between the theoretical behavior of the ABC posterior and that of the ABC-based predictive; ii) the use …
Predicting extreme events is important in many applications in risk analysis. The extreme-value theory suggests modelling extremes by max-stable distributions. The Bayesian approach provides a natural framework for statistical prediction. Although various Bayesian inferential procedures have been proposed in the literature of univariate extremes and some for multivariate extremes, the study of their asymptotic properties has been left largely untouched. In this paper we focus on a semiparatric Bayesian method for estimating max-stable distributions in arbitrary dimension. We establish consiste…
Accelerated life testing (ALT) is typically used to assess the reliability of material's lifetime under desired stress levels. Recent advances in material engineering have made a variety of material alternatives readily available. To identify the most reliable material setting with efficient experimental design, a sequential test planning strategy is preferred. To guarantee a tractable statistical mechanism for information collection and update, we develop explicit model parameter update formulas via approximate Bayesian inference. Theories show that our explicit update formulas give consisten…
Implementing Bayesian inference is often computationally challenging in applications involving complex models, and sometimes calculating the likelihood itself is difficult. Synthetic likelihood is one approach for carrying out inference when the likelihood is intractable, but it is straightforward to simulate from the model. The method constructs an approximate likelihood by taking a vector summary statistic as being multivariate normal, with the unknown mean and covariance matrix estimated by simulation for any given parameter value. Our article makes three contributions. The first shows that…
The construction of decision-theoretic Bayesian designs for realistically-complex nonlinear models is computationally challenging, as it requires the optimization of analytically intractable expected utility functions over high-dimensional design spaces. We provide the most general solution to date for this problem through a novel approximate coordinate exchange algorithm. This methodology uses a Gaussian process emulator to approximate the expected utility as a function of a single design coordinate in a series of conditional optimization steps. It has flexibility to address problems for any …
A major issue of extreme value analysis is the determination of the shape parameter $ξ$ common to Generalized Extreme Value (GEV) and Generalized Pareto (GP) distributions, which drives the tail behavior, and is of major impact on the estimation of return levels and periods. Many practitioners make the choice of a Bayesian framework to conduct this assessment for accounting of parametric uncertainties, which are typically high in such analyses characterized by a low number of observations. Nonetheless, such approaches can provide large credibility domains for $ξ$, including negative and positi…
Data-driven anomaly detection methods typically build a model for the normal behavior of the target system, and score each data instance with respect to this model. A threshold is invariably needed to identify data instances with high (or low) scores as anomalies. This presents a practical limitation on the applicability of such methods, since most methods are sensitive to the choice of the threshold, and it is challenging to set optimal thresholds. We present a probabilistic framework to explicitly model the normal and anomalous behaviors and probabilistically reason about the data. An extrem…
We consider an input-to-response (ItR) system characterized by (1) parameterized input with a known probability distribution and (2) stochastic ItR function with heteroscedastic randomness. Our purpose is to efficiently quantify the extreme response probability when the ItR function is expensive to evaluate. The problem setup arises often in physics and engineering problems, with randomness in ItR coming from either intrinsic uncertainties (say, as a solution to a stochastic equation) or additional (critical) uncertainties that are not incorporated in a low-dimensional input parameter space (a…
