Model selection is an important activity in modern data analysis and the conventional Bayesian approach to this problem involves calculation of marginal likelihoods for different models, together with diagnostics which examine specific aspects of model fit. Calculating the marginal likelihood is a difficult computational problem. Our article proposes some extensions of the Laplace approximation for this task that are related to copula models and which are easy to apply. Variations which can be used both with and without simulation from the posterior distribution are considered, as well as use of the approximations with bridge sampling and in random effects models with a large number of latent variables. The use of a t-copula to obtain higher accuracy when multivariate dependence is not well captured by a Gaussian copula is also discussed.
Copulas provide an attractive approach for constructing multivariate distributions with flexible marginal distributions and different forms of dependences. Of particular importance in many areas is the possibility of explicitly forecasting the tail-dependences. Most of the available approaches are only able to estimate tail-dependences and correlations via nuisance parameters, but can neither be used for interpretation, nor for forecasting. Aiming to improve copula forecasting performance, we propose a general Bayesian approach for modeling and forecasting tail-dependences and correlations as …
In Chib (1995), a method for approximating marginal densities in a Bayesian setting is proposed, with one proeminent application being the estimation of the number of components in a normal mixture. As pointed out in Neal (1999) and Fruhwirth-Schnatter (2004), the approximation often fails short of providing a proper approximation to the true marginal densities because of the well-known label switching problem (Celeux et al., 2000). While there exist other alternatives to the derivation of approximate marginal densities, we reconsider the original proposal here and show as in Berkhof et al. (2…
Parametric conditional copula models allow the copula parameters to vary with a set of covariates according to an unknown calibration function. Flexible Bayesian inference for the calibration function of a bivariate conditional copula is proposed via a sparse Gaussian process (GP) prior distribution over the set of smooth calibration functions for the single index model (SIM). The estimation of parameters from the marginal distributions and the calibration function is done jointly via Markov Chain Monte Carlo sampling from the full posterior distribution. A new Conditional Cross Validated Pseu…
We consider multivariate copula-based stationary time-series under Gaussian subordination. Observed time series are subordinated to long-range dependent Gaussian processes and characterized by arbitrary marginal copula distributions. First of all, we establish limit theorems for the marginal and quantile marginal empirical processes of multivariate stationary long-range dependent sequences under Gaussian subordination. Furthermore, we establish the asymptotic behavior of sequential empirical copula processes under non-restrictive smoothness assumptions. The limiting processes in the case of lo…
Conditional copulas are flexible statistical tools that couple joint conditional and marginal conditional distributions. In a linear regression setting with more than one covariate and two dependent outcomes, we propose the use of additive models for conditional bivariate copula models and discuss computation and model selection tools for performing Bayesian inference. The method is illustrated using simulations and a real example.
In this manuscript, we consider a finite multivariate nonparametric mixture model where the dependence between the marginal densities is modeled using the copula device. Pseudo EM stochastic algorithms were recently proposed to estimate all of the components of this model under a location-scale constraint on the marginals. Here, we introduce a deterministic algorithm that seeks to maximize a smoothed semiparametric likelihood. No location-scale assumption is made about the marginals. The algorithm is monotonic in one special case, and, in another, leads to ``approximate monotonicity'' -- where…
Approximate Bayesian computation (ABC) refers to a family of inference methods used in the Bayesian analysis of complex models where evaluation of the likelihood is difficult. Conventional ABC methods often suffer from the curse of dimensionality, and a marginal adjustment strategy was recently introduced in the literature to improve the performance of ABC algorithms in high-dimensional problems. The marginal adjustment approach is extended using a Gaussian copula approximation. The method first estimates the bivariate posterior for each pair of parameters separately using a 2-dimensional Gaus…
The majority of model-based clustering techniques is based on multivariate Normal models and their variants. In this paper copulas are used for the construction of flexible families of models for clustering applications. The use of copulas in model-based clustering offers two direct advantages over current methods: i) the appropriate choice of copulas provides the ability to obtain a range of exotic shapes for the clusters, and ii) the explicit choice of marginal distributions for the clusters allows the modelling of multivariate data of various modes (either discrete or continuous) in a natur…
Deep neural network (DNN) regression models are widely used in applications requiring state-of-the-art predictive accuracy. However, until recently there has been little work on accurate uncertainty quantification for predictions from such models. We add to this literature by outlining an approach to constructing predictive distributions that are `marginally calibrated'. This is where the long run average of the predictive distributions of the response variable matches the observed empirical margin. Our approach considers a DNN regression with a conditionally Gaussian prior for the final layer…
In this paper, a Bayesian semiparametric copula approach is used to model the underlying multivariate distribution $F_{true}$. First, the Dirichlet process is constructed on the unknown marginal distributions of $F_{true}$. Then a Gaussian copula model is utilized to capture the dependence structure of $F_{true}$. As a result, a Bayesian multivariate normality test is developed by combining the relative belief ratio and the Energy distance. Several interesting theoretical results of the approach are derived. Finally, through several simulated examples and a real data set, the proposed approach…
Copula-based models provide a great deal of flexibility in modelling multivariate distributions, allowing for the specifications of models for the marginal distributions separately from the dependence structure (copula) that links them to form a joint distribution. Choosing a class of copula models is not a trivial task and its misspecification can lead to wrong conclusions. We introduce a novel class of grid-uniform copula functions, which is dense in the space of all continuous copula functions in a Hellinger sense. We propose a Bayesian model based on this class and develop an automatic Mar…
Identifiable Bayesian Deep Generative Copulas with Unknown Layer Widths for Data with Arbitrary Marginal Distributions
Department of Statistics and Data Science（统计与数据科学系）；Washington University in St. Louis（圣路易斯华盛顿大学）；Department of Statistics（统计系）；Columbia University（哥伦比亚大学）
In this article, we analyze perinatal data with birth weight (BW) as primarily interesting response variable. Gestational age (GA) is usually an important covariate and included in polynomial form. However, in opposition to this univariate regression, bivariate modeling of BW and GA is recommended to distinguish effects on each, on both, and between them. Rather than a parametric bivariate distribution, we apply conditional copula regression, where marginal distributions of BW and GA (not necessarily of the same form) can be estimated independently, and where the dependence structure is modele…
We utilize copulas to constitute a unified framework for constructing and optimizing variational proposals in hierarchical Bayesian models. For models with continuous and non-Gaussian hidden variables, we propose a semiparametric and automated variational Gaussian copula approach, in which the parametric Gaussian copula family is able to preserve multivariate posterior dependence, and the nonparametric transformations based on Bernstein polynomials provide ample flexibility in characterizing the univariate marginal posteriors.
Copulas allow to learn marginal distributions separately from the multivariate dependence structure (copula) that links them together into a density function. Vine factorizations ease the learning of high-dimensional copulas by constructing a hierarchy of conditional bivariate copulas. However, to simplify inference, it is common to assume that each of these conditional bivariate copulas is independent from its conditioning variables. In this paper, we relax this assumption by discovering the latent functions that specify the shape of a conditional copula given its conditioning variables We le…
We introduce a copula mixture model to perform dependency-seeking clustering when co-occurring samples from different data sources are available. The model takes advantage of the great flexibility offered by the copulas framework to extend mixtures of Canonical Correlation Analysis to multivariate data with arbitrary continuous marginal densities. We formulate our model as a non-parametric Bayesian mixture, while providing efficient MCMC inference. Experiments on synthetic and real data demonstrate that the increased flexibility of the copula mixture significantly improves the clustering and t…
Gaussian factor models have proven widely useful for parsimoniously characterizing dependence in multivariate data. There is a rich literature on their extension to mixed categorical and continuous variables, using latent Gaussian variables or through generalized latent trait models acommodating measurements in the exponential family. However, when generalizing to non-Gaussian measured variables the latent variables typically influence both the dependence structure and the form of the marginal distributions, complicating interpretation and introducing artifacts. To address this problem we prop…
An important feature of Bayesian statistics is the opportunity to do sequential inference: the posterior distribution obtained after seeing a dataset can be used as prior for a second inference. However, when Monte Carlo sampling methods are used for inference, we only have a set of samples from the posterior distribution. To do sequential inference, we then either have to evaluate the second posterior at only these locations and reweight the samples accordingly, or we can estimate a functional description of the posterior probability distribution from the samples and use that as prior for the…
Estimating the marginal and joint densities of the long-term average intakes of different dietary components is an important problem in nutritional epidemiology. Since these variables cannot be directly measured, data are usually collected in the form of 24-hour recalls of the intakes, which show marked patterns of conditional heteroscedasticity. Significantly compounding the challenges, the recalls for episodically consumed dietary components also include exact zeros. The problem of estimating the density of the latent long-time intakes from their observed measurement error contaminated proxi…
Our article addresses the problem of flexibly estimating a multivariate density while also attempting to estimate its marginals correctly. We do so by proposing two new estimators that try to capture the best features of mixture of normals and copula estimators while avoiding some of their weaknesses. The first estimator we propose is a mixture of normals copula model that is a flexible alternative to parametric copula models such as the normal and t copula. The second is a marginally adapted mixture of normals estimator that improves on the standard mixture of normals by using information con…
In this paper, we present the algorithm that alters the subset of marginals of multivariate standard distributed data into such modelled by an Archimedean copula. Proposed algorithm leaves a correlation matrix almost unchanged, but introduces a higher order correlation into a subset of marginals. Our data transformation algorithm can be used to analyse whether particular machine learning algorithm, especially a dimensionality reduction one, utilises higher order correlations or not. We present an exemplary application on two features selection algorithms, mention that features selection is one…
This paper addresses the problem of quantification and propagation of uncertainties associated with dependence modeling when data for characterizing probability models are limited. Practically, the system inputs are often assumed to be mutually independent or correlated by a multivariate Gaussian distribution. However, this subjective assumption may introduce bias in the response estimate if the real dependence structure deviates from this assumption. In this work, we overcome this limitation by introducing a flexible copula dependence model to capture complex dependencies. A hierarchical Baye…
In this paper, we compare two numerical methods for approximating the probability that the sum of dependent regularly varying random variables exceeds a high threshold under Archimedean copula models. The first method is based on conditional Monte Carlo. We present four estimators and show that most of them have bounded relative errors. The second method is based on analytical expressions of the multivariate survival or cumulative distribution functions of the regularly varying random variables and provides sharp and deterministic bounds of the probability of exceedance. We discuss implementat…
Learning the joint dependence of discrete variables is a fundamental problem in machine learning, with many applications including prediction, clustering and dimensionality reduction. More recently, the framework of copula modeling has gained popularity due to its modular parametrization of joint distributions. Among other properties, copulas provide a recipe for combining flexible models for univariate marginal distributions with parametric families suitable for potentially high dimensional dependence structures. More radically, the extended rank likelihood approach of Hoff (2007) bypasses le…
This article proposes a space-efficient approximation to empirical tail dependence coefficients of an indefinite bivariate stream of data. The approximation, which has stream-length invariant error bounds, utilises recent work on the development of a summary for bivariate empirical copula functions. The work in this paper accurately approximates a bivariate empirical copula in the tails of each marginal distribution, therefore modelling the tail dependence between the two variables observed in the data stream. Copulas evaluated at these marginal tails can be used to estimate the tail dependenc…
