With the advancements of computer architectures, the use of computational models proliferates to solve complex problems in many scientific applications such as nuclear physics and climate research. However, the potential of such models is often hindered because they tend to be computationally expensive and consequently ill-fitting for uncertainty quantification. Furthermore, they are usually not calibrated with real-time observations. We develop a computationally efficient algorithm based on variational Bayes inference (VBI) for calibration of computer models with Gaussian processes. Unfortunately, the speed and scalability of VBI diminishes when applied to the calibration framework with dependent data. To preserve the efficiency of VBI, we adopt a pairwise decomposition of the data likelihood using vine copulas that separate the information on dependence structure in data from their marginal distributions. We provide both theoretical and empirical evidence for the computational scalability of our methodology and describe all the necessary details for an efficient implementation of the proposed algorithm. We also demonstrate the opportunities given by our method for practitioners on a real data example through calibration of the Liquid Drop Model of nuclear binding energies.
Vine copulas allow to build flexible dependence models for an arbitrary number of variables using only bivariate building blocks. The number of parameters in a vine copula model increases quadratically with the dimension, which poses new challenges in high-dimensional applications. To alleviate the computational burden and risk of overfitting, we propose a modified Bayesian information criterion (BIC) tailored to sparse vine copula models. We show that the criterion can consistently distinguish between the true and alternative models under less stringent conditions than the classical BIC. The …
The majority of finite mixture models suffer from not allowing asymmetric tail dependencies within components and not capturing non-elliptical clusters in clustering applications. Since vine copulas are very flexible in capturing these types of dependencies, we propose a novel vine copula mixture model for continuous data. We discuss the model selection and parameter estimation problems and further formulate a new model-based clustering algorithm. The use of vine copulas in clustering allows for a range of shapes and dependency structures for the clusters. Our simulation experiments illustrate…
We propose a class of dynamic vine copula models. This is an extension of static vine copulas and a generalization of dynamic C-vine and D-vine copulas studied by Almeida et al (2016) and Goel and Mehra (2019). Within this class, we allow for time-varying dependence by driving the vine copula parameters with latent AR(1) processes. This modeling approach is very flexible but estimation is not straightforward due to the high-dimensional parameter space. We propose a Bayesian estimation approach, which relies on a novel approximation of the posterior distribution. This approximation allows to us…
Copulas allow to learn marginal distributions separately from the multivariate dependence structure (copula) that links them together into a density function. Vine factorizations ease the learning of high-dimensional copulas by constructing a hierarchy of conditional bivariate copulas. However, to simplify inference, it is common to assume that each of these conditional bivariate copulas is independent from its conditioning variables. In this paper, we relax this assumption by discovering the latent functions that specify the shape of a conditional copula given its conditioning variables We le…
Vine Copulas as Differentiable Computational Graphs
National University of Singapore（新加坡国立大学）；University of Applied Sciences Western Switzerland (HES-SO)（瑞士西部应用科学与艺术大学）；LMU Munich（慕尼黑大学）；Munich Center for Machine Learning（慕尼黑机器学习中心）
Bayesian model selection of vine copulas: a loss-based perspective
Università Cattolica del Sacro Cuore（天主教撒科鲁尔圣心大学）；University of Turin（都灵大学）；University of Exeter（埃克塞特大学）；King’s College London（伦敦国王学院）；Duke Kunshan University（杜克昆山大学）
Regular vine copulas can describe a wider array of dependency patterns than the multivariate Gaussian copula or the multivariate Student's t copula. This paper presents two contributions related to model selection of regular vine copulas. First, our pair copula family selection procedure extends existing Bayesian family selection methods by allowing pair families to be chosen from an arbitrary set of candidate families. Second, our method represents the first Bayesian model selection approach to include the regular vine density construction in its scope of inference. The merits of our approach…
While there is considerable effort to identify signaling pathways using linear Gaussian Bayesian networks from data, there is less emphasis of understanding and quantifying conditional densities and probabilities of nodes given its parents from the identifed Bayesian network. Most graphical models for continuous data assume a multivariate Gaussian distribution, which might be too restrictive. We re-analyse data from an experimental setting considered in Sachs et al. (2005) to illustrate the effects of such restrictions. For this we propose a novel non Gaussian nonlinear structural equation mod…
We propose a model for unbalanced longitudinal data, where the univariate margins can be selected arbitrarily and the dependence structure is described with the help of a D-vine copula. We show that our approach is an extremely flexible extension of the widely used linear mixed model if the correlation is homogeneous over the considered individuals. As an alternative to joint maximum-likelihood a sequential estimation approach for the D-vine copula is provided and validated in a simulation study. The model can handle missing values without being forced to discard data. Since conditional distri…
The increasing penetration of renewable energy along with the variations of the loads bring large uncertainties in the power system states that are threatening the security of power system planning and operation. Facing these challenges, this paper proposes a cost-effective, nonparametric method to quantify the impact of uncertain power injections on the load margins. First, we propose to generate system uncertain inputs via a novel vine copula due to its capability in simulating complex multivariate highly dependent model inputs. Furthermore, to reduce the prohibitive computational time requi…
In recent years, conditional copulas, that allow dependence between variables to vary according to the values of one or more covariates, have attracted increasing attention. In high dimension, vine copulas offer greater flexibility compared to multivariate copulas, since they are constructed using bivariate copulas as building blocks. In this paper we present a novel inferential approach for multivariate distributions, which combines the flexibility of vine constructions with the advantages of Bayesian nonparametrics, not requiring the specification of parametric families for each pair copula.…
Copula models have become one of the most widely used tools in the applied modelling of multivariate data. Similarly, Bayesian methods are increasingly used to obtain efficient likelihood-based inference. However, to date, there has been only limited use of Bayesian approaches in the formulation and estimation of copula models. This article aims to address this shortcoming in two ways. First, to introduce copula models and aspects of copula theory that are especially relevant for a Bayesian analysis. Second, to outline Bayesian approaches to formulating and estimating copula models, and their …
In this paper we propose a flexible class of multivariate nonlinear non-Gaussian state space models, based on copulas. More precisely, we assume that the observation equation and the state equation are defined by copula families that are not necessarily equal. For each time point, the resulting model can be described by a C-vine copula truncated after the first tree, where the root node is represented by the latent state. Inference is performed within the Bayesian framework, using the Hamiltonian Monte Carlo method, where a further D-vine truncated after the first tree is used as prior distrib…
Cluster-specific ranking and variable importance for Scottish regional deprivation via vine mixtures
Delft Institute of Applied Mathematics, Delft University of Technology（代尔夫特理工大学）；School of Mathematics and Maxwell Institute for Mathematical Sciences, University of Edinburgh（爱丁堡大学）；Department of Mathematics, School of Computation, Information and Technology, Technical University of Munich（慕尼黑工业大学）
We introduce a new goodness-of-fit test for regular vine (R-vine) copula models, a flexible class of multivariate copulas based on a pair-copula construction (PCC). The test arises from the information matrix ratio. The corresponding test statistic is derived and its asymptotic normality is proven. The test's power is investigated and compared to 14 other goodness-of-fit tests, adapted from the bivariate copula case, in a high dimensional setting. The extensive simulation study shows the excellent performance with respect to size and power as well as the superiority of the information matrix r…
Systems subject to uncertain inputs produce uncertain responses. Uncertainty quantification (UQ) deals with the estimation of statistics of the system response, given a computational model of the system and a probabilistic model of its inputs. In engineering applications it is common to assume that the inputs are mutually independent or coupled by a Gaussian or elliptical dependence structure (copula). In this paper we overcome such limitations by modelling the dependence structure of multivariate inputs as vine copulas. Vine copulas are models of multivariate dependence built from simpler pai…
We introduce a new goodness-of-fit test for regular vine (R-vine) copula models. R-vine copulas are a very flexible class of multivariate copulas based on a pair-copula construction (PCC). The test arises from the information matrix equality and specification test proposed by White (1982) and extends the goodness-of-fit test for copulas introduced by Huang and Prokhorov (2011). The corresponding critical value can be approximated by asymptotic theory or simulation. The simulation based test shows excellent performance with regard to observed size and power in an extensive simulation study, whi…
Parametric conditional copula models allow the copula parameters to vary with a set of covariates according to an unknown calibration function. Flexible Bayesian inference for the calibration function of a bivariate conditional copula is proposed via a sparse Gaussian process (GP) prior distribution over the set of smooth calibration functions for the single index model (SIM). The estimation of parameters from the marginal distributions and the calibration function is done jointly via Markov Chain Monte Carlo sampling from the full posterior distribution. A new Conditional Cross Validated Pseu…
An important feature of Bayesian statistics is the opportunity to do sequential inference: the posterior distribution obtained after seeing a dataset can be used as prior for a second inference. However, when Monte Carlo sampling methods are used for inference, we only have a set of samples from the posterior distribution. To do sequential inference, we then either have to evaluate the second posterior at only these locations and reweight the samples accordingly, or we can estimate a functional description of the posterior probability distribution from the samples and use that as prior for the…
We demonstrate how the uncertainty of parameter point estimates can be assessed in a maximum likelihood framework in order to prevent overfitting and erroneous detection of time-inhomogeneity. The class of models we consider are regular vine (R-vine) copula models, for which we describe a new algorithm for the exact computation of the score function and observed information. R-vine copulas constitute a flexible class of dependence models which are constructed hierarchically from bivariate copulas as building blocks only, and our algorithm exploits the hierarchical nature for subsequent computa…
One of the main challenges in current systems neuroscience is the analysis of high-dimensional neuronal and behavioral data that are characterized by different statistics and timescales of the recorded variables. We propose a parametric copula model which separates the statistics of the individual variables from their dependence structure, and escapes the curse of dimensionality by using vine copula constructions. We use a Bayesian framework with Gaussian Process (GP) priors over copula parameters, conditioned on a continuous task-related variable. We validate the model on synthetic data and c…
For a particular disease there may be two diagnostic tests developed, where each of the tests is subject to several studies. A quadrivariate generalized linear mixed model (GLMM) has been recently proposed to joint meta-analyse and compare two diagnostic tests. We propose a D-vine copula mixed model for joint meta-analysis and comparison of two diagnostic tests. Our general model includes the quadrivariate GLMM as a special case and can also operate on the original scale of sensitivities and specificities. The method allows the direct calculation of sensitivity and specificity for each test, a…
Misperceptions about extreme dependencies between different financial assets have been an im- portant element of the recent financial crisis. This paper studies inhomogeneity in dependence structures using Markov switching regular vine copulas. These account for asymmetric depen- dencies and tail dependencies in high dimensional data. We develop methods for fast maximum likelihood as well as Bayesian inference. Our algorithms are validated in simulations and applied to financial data. We find that regime switches are present in the dependence structure of various data sets and show that regime…
Deep neural network (DNN) regression models are widely used in applications requiring state-of-the-art predictive accuracy. However, until recently there has been little work on accurate uncertainty quantification for predictions from such models. We add to this literature by outlining an approach to constructing predictive distributions that are `marginally calibrated'. This is where the long run average of the predictive distributions of the response variable matches the observed empirical margin. Our approach considers a DNN regression with a conditionally Gaussian prior for the final layer…
We consider the problem of modeling the dependence among many time series. We build high dimensional time-varying copula models by combining pair-copula constructions (PCC) with stochastic autoregressive copula (SCAR) models to capture dependence that changes over time. We show how the estimation of this highly complex model can be broken down into the estimation of a sequence of bivariate SCAR models, which can be achieved by using the method of simulated maximum likelihood. Further, by restricting the conditional dependence parameter on higher cascades of the PCC to be constant, we can great…
In this paper, we concentrate on new methodologies for copulas introduced and developed by Joe, Cooke, Bedford, Kurowica, Daneshkhah and others on the new class of graphical models called vines as a way of constructing higher dimensional distributions. We develop the approximation method presented by Bedford et al (2012) at which they show that any $n$-dimensional copula density can be approximated arbitrarily well pointwise using a finite parameter set of 2-dimensional copulas in a vine or pair-copula construction. Our constructive approach involves the use of minimum information copulas that…
A2 Copula-Driven Spatial Bayesian Neural Network For Modeling Non-Gaussian Dependence: A Simulation Study
Department of Mathematics, University of Louisiana at Lafayette（数学系，路易斯安那州立大学拉斐特分校）；Department of Physical Sciences & Mathematics, West Liberty University（物理科学与数学系，韦斯特利比大学）；Department of Mathematics and Statistics, Minnesota State University（数学与统计学系，明尼苏达州立大学）；Department of Statistics, formerly of Presidency College, Kolkata, India（统计学系，原加尔各答 Presidency College）；Department of Kinesiology, University of Louisiana at Lafayette（运动科学系，路易斯安那州立大学拉斐特分校）；Department of Mathematics and Statistics, Texas Tech University（数学与统计学系，德克萨斯技术大学）；The Office of Vice President for Research, Innovation, and Economic Development, University of Louisiana at Lafayette（研究、创新与经济发展副总监办公室，路易斯安那州立大学拉斐特分校）
Key to effective generic, or "black-box", variational inference is the selection of an approximation to the target density that balances accuracy and speed. Copula models are promising options, but calibration of the approximation can be slow for some choices. Smith et al. (2020) suggest using tractable and scalable "implicit copula" models that are formed by element-wise transformation of the target parameters. We propose an adjustment to these transformations that make the approximation invariant to the scale and location of the target density. We also show how a sub-class of elliptical copu…
