Consider a multinomial regression model where the response, which indicates a unit's membership in one of several possible unordered classes, is associated with a set of predictor variables. Such models typically involve a matrix of regression coefficients, with the $(j,k)$ element of this matrix modulating the effect of the $k$th predictor on the propensity of the unit to belong to the $j$th class. Thus, a supposition that only a subset of the available predictors are associated with the response corresponds to some of the columns of the coefficient matrix being zero. Under the Bayesian paradigm, the subset of predictors which are associated with the response can be treated as an unknown parameter, leading to typical Bayesian model selection and model averaging procedures. As an alternative, we investigate model selection and averaging, whereby a subset of individual elements of the coefficient matrix are zero. That is, the subset of predictors associated with the propensity to belong to a class varies with the class. We refer to this as class-specific predictor selection. We argue that such a scheme can be attractive on both conceptual and computational grounds.
MPBART - Multinomial Probit Bayesian Additive Regression Trees
本文提出了多项式概率贝叶斯加性回归树（MPBART）作为BART-贝叶斯加性回归树的多项式概率扩展（Chipman et al（2010））。MPBART是灵活的，允许包括预测，描述观察单位以及可用的选择方案。通过两个模拟研究和四个真实数据的例子，我们表明，MPBART具有很好的预测性能相比，其他离散选择和多类分类方法。为了实现MPBART，我们开发了一个R包mpbart，可以从CRAN仓库免费获得。
The Dirichlet-multinomial (DM) distribution plays a fundamental role in modern statistical methodology development and application. Recently, the DM distribution and its variants have been used extensively to model multivariate count data generated by high-throughput sequencing technology in omics research due to its ability to accommodate the compositional structure of the data as well as overdispersion. A major limitation of the DM distribution is that it is unable to handle excess zeros typically found in practice which may bias inference. To fill this gap, we propose a novel Bayesian zero-…
Multinomial probit models are routinely-implemented representations for learning how the class probabilities of categorical response data change with p observed predictors. Although several frequentist methods have been developed for estimation, inference and classification within such a class of models, Bayesian inference is still lagging behind. This is due to the apparent absence of a tractable class of conjugate priors, that may facilitate posterior inference on the multinomial probit coefficients. Such an issue has motivated increasing efforts toward the development of effective Markov ch…
Multinomial probit (mnp) models are fundamental and widely-applied regression models for categorical data. Fasano and Durante (2022) proved that the class of unified skew-normal distributions is conjugate to several mnp sampling models. This allows to develop Monte Carlo samplers and accurate variational methods to perform Bayesian inference. In this paper, we adapt the abovementioned results for a popular special case: the discrete-choice mnp model under zero mean and independent Gaussian priors. This allows to obtain simplified expressions for the parameters of the posterior distribution and…
A broad class of models that routinely appear in several fields can be expressed as partially or fully discretized Gaussian linear regressions. Besides including basic Gaussian response settings, this class also encompasses probit, multinomial probit and tobit regression, among others, thereby yielding to one of the most widely-implemented families of models in applications. The relevance of such representations has stimulated decades of research in the Bayesian field, mostly motivated by the fact that, unlike for Gaussian linear regression, the posterior distribution induced by such models do…
Johns Hopkins University（约翰斯·霍普金斯大学）；Brown University（布朗大学）；University of Florida（佛罗里达大学）；Moi University（莫伊大学）
The multinomial probit Bayesian additive regression trees (MPBART) framework was proposed by Kindo et al. (KD), approximating the latent utilities in the multinomial probit (MNP) model with BART (Chipman et al. 2010). Compared to multinomial logistic models, MNP does not assume independent alternatives and the correlation structure among alternatives can be specified through multivariate Gaussian distributed latent utilities. We introduce two new algorithms for fitting the MPBART and show that the theoretical mixing rates of our proposals are equal or superior to the existing algorithm in KD. …
On Bayesian robust regression with diverging number of predictors
研究了预测变量数与观测变量数以相似速率增长时的稳健回归模型。最近，几位作者开发了这种状态下的M估计量理论[El Karoui等人，2013，Bean等人，2013，Donoho and Montanari，2013].由于M-估计无法成功地估计系数向量的欧氏范数，我们考虑了这个模型的贝叶斯框架。我们建议一个两个组件的混合物的正常的系数之前，并开发一个吉布斯采样程序从相关的后验分布采样，同时利用一个规模的混合正态表示的误差分布。不同于M-估计，建议的贝叶斯估计是一致的，在欧几里德范数意义。仿真结果表明，贝叶斯估计优于传统的估计方法。
This paper studies posterior contraction rates in multi-category logit models with priors incorporating group sparse structures. We consider a general class of logit models that includes the well-known multinomial logit models as a special case. Group sparsity is useful when predictor variables are naturally clustered and particularly useful for variable selection in the multinomial logit models. We provide a unified platform for posterior contraction rates of group-sparse logit models that include binary logistic regression under individual sparsity. No size restriction is directly imposed on…
Categorizing individual cells into one of many known cell type categories, also known as cell type annotation, is a critical step in the analysis of single-cell genomics data. The current process of annotation is time-intensive and subjective, which has led to different studies describing cell types with labels of varying degrees of resolution. While supervised learning approaches have provided automated solutions to annotation, there remains a significant challenge in fitting a unified model for multiple datasets with inconsistent labels. In this article, we propose a new multinomial logistic…
$\ell_1$-penalized Multinomial Regression: Estimation, inference, and prediction, with an application to risk factor identification for different dementia subtypes
Multinomial logistic regression models allow one to predict the risk of a categorical outcome with more than 2 categories. When developing such a model, researchers should ensure the number of participants (n) is appropriate relative to the number of events (E.k) and the number of predictor parameters (p.k) for each category k. We propose three criteria to determine the minimum n required in light of existing criteria developed for binary outcomes. The first criteria aims to minimise the model overfitting. The second aims to minimise the difference between the observed and adjusted R2 Nagelker…
This paper considers the Liu estimator in the multinomial logistic regression model. We propose some different estimators of the biasing parameter. The mean square error (MSE) is considered as the performance criterion. In order to compare the performance of the estimators, we performed a Monte Carlo simulation study. According to the results of the simulation study, we found that increasing the correlation between the independent variables and the number of regressors has a negative effect on the MSE. However, when the sample size increases the MSE decreases even when the correlation between …
Comparison of probabilistic nowcasts and forecasts of SARS-CoV-2 variant proportions made by hierarchical multinomial linear regression models
Department of Mathematics and Statistics, University of Massachusetts, Amherst（马萨诸塞大学阿默斯特分校数学与统计系）；School of Public Health and Health Sciences, University of Massachusetts, Amherst（马萨诸塞大学阿默斯特分校公共卫生与健康科学学院）
Regression factor score predictors have the maximum factor score determinacy, i.e., the maximum correlation with the corresponding factor, but they do not have the same inter-correlations as the factors. As it might be useful to compute factor score predictors that have the same inter-correlations as the factors, correlation-preserving factor score predictors have been proposed. However, correlation-preserving factor score predictors have smaller correlations with the corresponding factors (factor score determinacy) than regression factor score predictors. Thus, higher factor score determinacy…
Brain segmentation at different levels is generally represented as hierarchical trees. Brain regional atrophy at specific levels was found to be marginally associated with Alzheimer's disease outcomes. In this study, we propose an L1-type regularization for predictors that follow a hierarchical tree structure. Considering a tree as a directed acyclic graph, we interpret the model parameters from a path analysis perspective. Under this concept, the proposed penalty regulates the total effect of each predictor on the outcome. With regularity conditions, it is shown that under the proposed regula…
Variable selection is an important statistical problem. This problem becomes more challenging when the candidate predictors are of mixed type (e.g. continuous and binary) and impact the response variable in nonlinear and/or non-additive ways. In this paper, we review existing variable selection approaches for the Bayesian additive regression trees (BART) model, a nonparametric regression model, which is flexible enough to capture the interactions between predictors and nonlinear relationships with the response. An emphasis of this review is on the capability of identifying relevant predictors.…
In this paper, we propose methods for functional predictor selection and the estimation of smooth functional coefficients simultaneously in a scalar-on-function regression problem under high-dimensional multivariate functional data setting. In particular, we develop two methods for functional group-sparse regression under a generic Hilbert space of infinite dimension. We show the convergence of algorithms and the consistency of the estimation and the selection (oracle property) under infinite-dimensional Hilbert spaces. Simulation studies show the effectiveness of the methods in both the selec…
We develop a novel Bayesian method to select important predictors in regression models with multiple responses of diverse types. A sparse Gaussian copula regression model is used to account for the multivariate dependencies between any combination of discrete and/or continuous responses and their association with a set of predictors. We utilize the parameter expansion for data augmentation strategy to construct a Markov chain Monte Carlo algorithm for the estimation of the parameters and the latent variables of the model. Based on a centered parametrization of the Gaussian latent variables, we…
