The multivariate probit is popular for modeling correlated binary data, with an attractive balance of flexibility and simplicity. However, considerable challenges remain in computation and in devising a clear statistical framework. Interest in the multivariate probit has increased in recent years. Current applications include genomics and precision medicine, where simultaneous modeling of multiple traits may be of interest, and computational efficiency is an important consideration. We propose a fast method for multivariate probit estimation via a two-stage composite likelihood. We explore computational and statistical efficiency, and note that the approach sets the stage for extensions beyond the purely binary setting.
This paper provides a closed form expression for the pairwise score vector for the multivariate ordered probit model. This result has several implications in likelihood-based inference. It is indeed used both to speed-up gradient based optimization routines for point estimation, and to provide a building block to compute standard errors and confidence intervals by means of the Godambe matrix.
Multivariate probit models (MPM) have the appealing feature of capturing some of the dependence structure between the components of multidimensional binary responses. The key for the dependence modelling is the covariance matrix of an underlying latent multivariate Gaussian. Most approaches to MLE in multivariate probit regression rely on MCEM algorithms to avoid computationally intensive evaluations of multivariate normal orthant probabilities. As an alternative to the much used Gibbs sampler a new SMC sampler for truncated multivariate normals is proposed. The algorithm proceeds in two stage…
Data reproducibility is a critical issue in all scientific experiments. In this manuscript, we consider the problem of quantifying the reproducibility of graphical measurements. We generalize the concept of image intra-class correlation coefficient (I2C2) and propose the concept of the graphical intra-class correlation coefficient (GICC) for such purpose. The concept of GICC is based on multivariate probit-linear mixed effect models. We will present a Markov Chain EM (MCEM) algorithm for estimating the GICC. Simulations results with varied settings are demonstrated and our method is applied to…
This article considers the problem of estimating a multivariate probit model in a panel data setting with emphasis on sampling a high-dimensional correlation matrix and improving the overall efficiency of the data augmentation approach. We reparameterise the correlation matrix in a principled way and then carry out efficient Bayesian inference using Hamiltonian Monte Carlo. We also propose a novel antithetic variable method to generate samples from the posterior distribution of the random effects and regression coefficients, resulting in significant gains in efficiency. We apply the methodolog…
Multinomial probit models are routinely-implemented representations for learning how the class probabilities of categorical response data change with p observed predictors. Although several frequentist methods have been developed for estimation, inference and classification within such a class of models, Bayesian inference is still lagging behind. This is due to the apparent absence of a tractable class of conjugate priors, that may facilitate posterior inference on the multinomial probit coefficients. Such an issue has motivated increasing efforts toward the development of effective Markov ch…
Inferring concerted changes among biological traits along an evolutionary history remains an important yet challenging problem. Besides adjusting for spurious correlation induced from the shared history, the task also requires sufficient flexibility and computational efficiency to incorporate multiple continuous and discrete traits as data size increases. To accomplish this, we jointly model mixed-type traits by assuming latent parameters for binary outcome dimensions at the tips of an unknown tree informed by molecular sequences. This gives rise to a phylogenetic multivariate probit model. Wi…
Correlated binary response data with covariates are ubiquitous in longitudinal or spatial studies. Among the existing statistical models the most well-known one for this type of data is the multivariate probit model, which uses a Gaussian link to model dependence at the latent level. However, a symmetric link may not be appropriate if the data are highly imbalanced. Here, we propose a multivariate skew-elliptical link model for correlated binary responses, which includes the multivariate probit model as a special case. Furthermore, we perform Bayesian inference for this new model and prove tha…
In this paper the properties of the maximum approximate composite marginal likelihood (MaCML) approach to the estimation of multinomial probit models (MNP) proposed by Chandra Bhat and coworkers is investigated in finite samples as well as with respect to asymptotic properties. Using a small illustration example it is proven that the approach does not necessarily lead to consistent estimators for four different types of approximation of the Gaussian cumulative distribution function (including the Solow-Joe approach proposed by Bhat). It is shown that the bias of parameter estimates can be subs…
The analysis of large scale medical claims data has the potential to improve quality of care by generating insights which can be used to create tailored medical programs. In particular, the multivariate probit model can be used to investigate the correlation between multiple binary responses of interest in such data, e.g. the presence of multiple chronic conditions. Bayesian modeling is well suited to such analyses because of the automatic uncertainty quantification provided by the posterior distribution. A complicating factor is that large medical claims datasets often do not fit in memory, w…
It has become increasingly common to collect high-dimensional binary response data; for example, with the emergence of new sampling techniques in ecology. In smaller dimensions, multivariate probit (MVP) models are routinely used for inferences. However, algorithms for fitting such models face issues in scaling up to high dimensions due to the intractability of the likelihood, involving an integral over a multivariate normal distribution having no analytic form. Although a variety of algorithms have been proposed to approximate this intractable integral, these approaches are difficult to imple…
Marginally specified models have recently become a popular tool for discrete longitudinal data analysis. Nonetheless, they introduce complex constraint equations and model fitting algorithms. Moreover, there is a lack of available software to fit these models. In this paper, we propose a three-level marginally specified model for analysis of multivariate longitudinal binary response data. The implicit function theorem is introduced to approximately solve the marginal constraint equations explicitly. Furthermore, the use of \textit{probit} link enables direct solutions to the convolution equati…
Composite likelihood estimation has an important role in the analysis of multivariate data for which the full likelihood function is intractable. An important issue in composite likelihood inference is the choice of the weights associated with lower-dimensional data sub-sets, since the presence of incompatible sub-models can deteriorate the accuracy of the resulting estimator. In this paper, we introduce a new approach for simultaneous parameter estimation by tilting, or re-weighting, each sub-likelihood component called discriminative composite likelihood estimation (D-McLE). The data-adaptiv…
Convex Reparameterization and Self-Concordant Algorithms for Multivariate Regression with Covariance Estimation
School of Statistics, University of Minnesota Twin Cities（明尼苏达大学双城分校统计学院）；Department of Applied Mathematics & Statistics, Stony Brook University（石溪大学应用数学与统计系）
Statistical modeling of multivariate and spatial extreme events has attracted broad attention in various areas of science. Max-stable distributions and processes are the natural class of models for this purpose, and many parametric families have been developed and successfully applied. Due to complicated likelihoods, the efficient statistical inference is still an active area of research, and usually composite likelihood methods based on bivariate densities only are used. Thibaud et al. (2016, Ann. Appl. Stat., to appear) use a Bayesian approach to fit a Brown--Resnick process to extreme tempe…
Multivariate categorical data are routinely collected in many application areas. As the number of cells in the table grows exponentially with the number of variables, many or even most cells will contain zero observations. This severe sparsity motivates appropriate statistical methodologies that effectively reduce the number of free parameters, with penalized log-linear models and latent structure analysis being popular options. This article proposes a fundamentally new class of methods, which we refer to as Mixture of Log Linear models (mills). Combining latent class analysis and log-linear m…
Expectation propagation is a general prescription for approximation of integrals in statistical inference problems. Its literature is mainly concerned with Bayesian inference scenarios. However, expectation propagation can also be used to approximate integrals arising in frequentist statistical inference. We focus on likelihood-based inference for binary response mixed models and show that fast and accurate quadrature-free inference can be realized for the probit link case with multivariate random effects and higher levels of nesting. The approach is supported by asymptotic theory in which exp…
The method of generalized estimating equations (GEE) is popular in the biostatistics literature for analyzing longitudinal binary and count data. It assumes a generalized linear model (GLM) for the outcome variable, and a working correlation among repeated measurements. In this paper, we introduce a viable competitor: the weighted scores method for GLM margins. We weight the univariate score equations using a working discretized multivariate normal model that is a proper multivariate model. Since the weighted scores method is a parametric method based on likelihood, we propose composite likeli…
Estimation of Linear models from Coarsened Observations Estimation of Linear models Estimation from Coarsened Observations A Method of Moments Approach
Max-stable random fields provide canonical models for the dependence of multivariate extremes. Inference with such models has been challenging due to the lack of tractable likelihoods. In contrast, the finite dimensional cumulative distribution functions (CDFs) are often readily available and natural to work with. Motivated by this fact, in this work we develop an M-estimation framework for max-stable models based on the continuous ranked probability score (CRPS) of multivariate CDFs. We start by establishing conditions for the consistency and asymptotic normality of the CRPS-based estimators …
The last decade has seen max-stable processes emerge as a common tool for the statistical modeling of spatial extremes. However, their application is complicated due to the unavailability of the multivariate density function, and so likelihood-based methods remain far from providing a complete and flexible framework for inference. In this article we develop inferentially practical, likelihood-based methods for fitting max-stable processes derived from a composite-likelihood approach. The procedure is sufficiently reliable and versatile to permit the simultaneous modeling of marginal and depend…
