In the analysis of count data often the equidispersion assumption is not suitable, hence the Poisson regression model is inappropriate. As a generalization of the Poisson distribution, the COM-Poisson distribution can deal with under-, equi- and overdispersed count data. It is a member of the exponential family of distributions and has well known special cases. In spite of the nice properties of the COM-Poisson distribution, its location parameter does not correspond to the expectation, which complicates the interpretation of regression models. In this paper, we propose a straightforward reparametrization of the COM-Poisson distribution based on an approximation to the expectation of this distribution. The main advantage of our new parametrization is the straightforward interpretation of the regression coefficients in terms of the expectation, as usual in the context of generalized linear models. Furthermore, the estimation and inference for the new COM-Poisson regression model can be done based on the likelihood paradigm. We carried out simulation studies to verify the finite sample properties of the maximum likelihood estimators. The results from our simulation study show that the maximum likelihood estimators are unbiased and consistent for both regression and dispersion parameters. We observed that the empirical correlation between the regression and dispersion parameter estimators is close to zero, which suggests that these parameters are orthogonal. We illustrate the application of the proposed model through the analysis of three data sets with over-, under- and equidispersed count data. The study of distribution properties through a consideration of dispersion, zero-inflated and heavy tail indexes, together with the results of data analysis show the flexibility over standard approaches.
In this paper, we propose improved estimation method for logistic regression based on subsamples taken according the optimal subsampling probabilities developed in Wang et al. 2018 Both asymptotic results and numerical results show that the new estimator has a higher estimation efficiency. We also develop a new algorithm based on Poisson subsampling, which does not require to approximate the optimal subsampling probabilities all at once. This is computationally advantageous when available random-access memory is not enough to hold the full data. Interestingly, asymptotic distributions also sho…
We consider the non-parametric Poisson regression problem where the integer valued response $Y$ is the realization of a Poisson random variable with parameter $λ(X)$. The aim is to estimate the functional parameter $λ$ from independent or weakly dependent observations $(X_1,Y_1),\ldots,(X_n,Y_n)$ in a random design framework. First we determine upper risk bounds for projection estimators on finite dimensional subspaces under mild conditions. In the case of Sobolev ellipsoids the obtained rates of convergence turn out to be optimal. The main part of the paper is devoted to the construction of a…
INTRODUCTION: Wald's, the likelihood ratio (LR) and Rao's score tests and their corresponding confidence intervals (CIs), are the three most common estimators of parameters of Generalized Linear Models. On finite samples, these estimators are biased. The objective of this work is to analyze the coverage errors of the CI estimators in small samples for the log-Poisson model (i.e. estimation of incidence rate ratio) with innovative evaluation criteria, taking in account the overestimation/underestimation unbalance of coverage errors and the variable inclusion rate and follow-up in epidemiologica…
In the era of open data, Poisson and other count regression models are increasingly important. Still, conventional Poisson regression has remaining issues in terms of identifiability and computational efficiency. Especially, due to an identification problem, Poisson regression can be unstable for small samples with many zeros. Provided this, we develop a closed-form inference for an over-dispersed Poisson regression including Poisson additive mixed models. The approach is derived via mode-based log-Gaussian approximation. The resulting method is fast, practical, and free from the identificatio…
Motivated by the current Coronavirus Disease (COVID-19) pandemic, which is due to the SARS-CoV-2 virus, and the important problem of forecasting daily deaths and cumulative deaths, this paper examines the construction of prediction regions or intervals under the Poisson regression model and for an over-dispersed Poisson regression model. For the Poisson regression model, several prediction regions are developed and their performance are compared through simulation studies. The methods are applied to the problem of forecasting daily and cumulative deaths in the United States (US) due to COVID-1…
Compared to the conditional mean as a simple point estimator, the conditional density function is more informative to describe the distributions with multi-modality, asymmetry or heteroskedasticity. In this paper, we propose a novel parametric conditional density estimation method by showing the connection between the general density and the likelihood function of inhomogeneous Poisson process models. The maximum likelihood estimates can be obtained via weighted logistic regressions, and the computation can be significantly relaxed by combining a block-wise alternating maximization scheme and …
Beta-binomial/gamma-Poisson regression models for repeated counts with random parameters
β-二项/泊松模型已被许多作者用来模拟多变量计数数据。Lora和Singer（Statistics in Medicine，2008）扩展了此类模型，以适应重复的多变量计数数据，其中二项式分量过度分散。为了克服该模型的一些局限性，我们考虑了β-二项式/γ-泊松替代方案，该方案还允许泊松计数之间的过度分散和不同的协方差。我们使用Newton-Raphson算法获得参数的最大似然估计，并在一个实际例子中比较两种模型。
The robust Poisson method is becoming increasingly popular when estimating the association of exposures with a binary outcome. Unlike the logistic regression model, the robust Poisson method yields results that can be interpreted as risk or prevalence ratios. In addition, it does not suffer from frequent non-convergence problems like the most common implementations of maximum likelihood estimators of the log-binomial model. However, using a Poisson distribution to model a binary outcome may seem counterintuitive. Methodological papers have often presented this as a good approximation to the mo…
The University of California, San Francisco（加州大学旧金山分校）；Tongji University（同济大学）；Syracuse University（雪城大学）
We study estimation and testing in the Poisson regression model with noisy high dimensional covariates, which has wide applications in analyzing noisy big data. Correcting for the estimation bias due to the covariate noise leads to a non-convex target function to minimize. Treating the high dimensional issue further leads us to augment an amenable penalty term to the target function. We propose to estimate the regression parameter through minimizing the penalized target function. We derive the L1 and L2 convergence rates of the estimator and prove the variable selection consistency. We further…
Being in the era of big data, modeling and prediction of count data have become significantly important in many fields including health, finance, social, etc. Although linear Poisson regression has been widely used to model count and rate data, it might not be always suitable as it cannot capture some inherent variability within complex data. In this study, we introduce a probabilistically driven nonlinear Poisson regression model with Bayesian artificial neural networks (ANN) to model count or rate data. This new nonlinear Poisson regression model developed with Bayesian ANN provides higher p…
This article is devoted to the forecast of the Africa Cup of Nations 2019 football tournament. It is based on a Poisson regression model that includes the Elo points of the participating teams as covariates and incorporates differences of team-specific skills. The proposed model allows predictions in terms of probabilities in order to quantify the chances for each team to reach a certain stage of the tournament. Monte Carlo simulations are used to estimate the outcome of each single match of the tournament and hence to simulate the whole tournament itself. The model is fitted on all football g…
The premise of independence among subjects in the same cluster/group often fails in practice, and models that rely on such untenable assumption can produce misleading results. To overcome this severe deficiency, we introduce a new regression model to handle overdispersed and correlated clustered counts. To account for correlation within clusters, we propose a Poisson regression model where the observations within the same cluster are driven by the same latent random effect that follows the Birnbaum-Saunders distribution with a parameter that controls the strength of dependence among the indivi…
We present a new modeling technique for solving the problem of ecological inference, in which individual-level associations are inferred from labeled data available only at the aggregate level. We model aggregate count data as arising from the Poisson binomial, the distribution of the sum of independent but not identically distributed Bernoulli random variables. We relate individual-level probabilities to individual covariates using both a logistic regression and a neural network. A normal approximation is derived via the Lyapunov Central Limit Theorem, allowing us to efficiently fit these mod…
We consider a problem of ecological inference, in which individual-level covariates are known, but labeled data is available only at the aggregate level. The intended application is modeling voter preferences in elections. In Rosenman and Viswanathan (2018), we proposed modeling individual voter probabilities via a logistic regression, and posing the problem as a maximum likelihood estimation for the parameter vector beta. The likelihood is a Poisson binomial, the distribution of the sum of independent but not identically distributed Bernoulli variables, though we approximate it with a heteros…
We study the problem of learning generalized linear models under adversarial corruptions. We analyze a classical heuristic called the iterative trimmed maximum likelihood estimator which is known to be effective against label corruptions in practice. Under label corruptions, we prove that this simple estimator achieves minimax near-optimal risk on a wide range of generalized linear models, including Gaussian regression, Poisson regression and Binomial regression. Finally, we extend the estimator to the more challenging setting of label and covariate corruptions and demonstrate its robustness a…
Premier League is known as one of the most competitive football league in the world, hence there are many goals are scored here every match. Which are the factors that affect to the number of goal scored in each match? We use Poisson regression to find out the relation between many factors as shots on target, corners, red cards, to the goals home team can score in their match.
This research paper explores application of multivariable regression models using only reservoir temperatures for predicting oil and gas production in a Toe-Heal-Air-Injection (THAI) enhanced oil recovery process. This paper discusses effects of statistical interaction between thermocouples by using second degree quadratic polynomials, which showed significant production forecast accuracy. Interactions among thermocouples statistically include temperature of larger reservoir areas, hence improving the predictive models. The interaction of two thermocouples can be interpreted as temperature gra…
In wake of the Covid-19 pandemic, 2019-2020 soccer seasons across the world were postponed and eventually made up during the summer months of 2020. Researchers from a variety of disciplines jumped at the opportunity to compare the rescheduled games, played in front of empty stadia, to previous games, played in front of fans. To date, most of this post-Covid soccer research has used linear regression models, or versions thereof, to estimate potential changes to the home advantage. But because soccer outcomes are non-linear, we argue that leveraging the Poisson distribution would be more appropr…
To understand potential encoding mechanism of motor cortical neurons for control commands during reach-to-grasp movements, experiments to record neuronal activities from primary motor cortical regions have been conducted in many research laboratories (for example, (7), (17)). The most popular approach in neuroscience community is to fit the Analysis of Variance (ANOVA) model using the firing rates of individual neurons. In addition to consider neural firing counts but also temporal intervals, (5) proposed to apply Analysis of Covariance (ANCOVA) model. Due to the nature of the data, in this pa…
This report is devoted to the forecast of the UEFA EURO 2020, Europe's continental football championship, taking place across Europe in June/July 2021. We present the simulation results for this tournament, where the simulations are based on a zero-inflated generalized Poisson regression model that includes the Elo points of the participating teams and the location of the matches as covariates and incorporates differences of team-specific skills. The proposed model allows predictions in terms of probabilities in order to quantify the chances for each team to reach a certain stage of the tourna…
Although basketball is a dynamic process sport, with 5 plus 5 players competing on both offense and defense simultaneously, learning some static information is predominant for professional players, coaches and team mangers. In order to have a deep understanding of field goal attempts among different players, we propose a zero inflated Poisson model with clustered regression coefficients to learn the shooting habits of different players over the court and the heterogeneity among them. Specifically, the zero inflated model recovers the large proportion of the court with zero field goal attempts,…
While the hurdle Poisson regression is a popular class of models for count data with excessive zeros, the link function in the binary component may be unsuitable for highly imbalanced cases. Ordinary Poisson regression is unable to handle the presence of dispersion. In this paper, we introduce Conway-Maxwell-Poisson (CMP) distribution and integrate use of flexible skewed Weibull link functions as better alternative. We take a fully Bayesian approach to draw inference from the underlying models to better explain skewness and quantify dispersion, with Deviance Information Criteria (DIC) used for…
Imputation of missing values is a strategy for handling non-responses in surveys or data loss in measurement processes, which may be more effective than ignoring them. When the variable represents a count, the literature dealing with this issue is scarce. Likewise, if problems of over- or under-dispersion are observed, generalisations of the Poisson distribution are recommended for carrying out imputation. In order to assess the performance of various regression models in the imputation of a discrete variable compared to classical counting models, this work presents a comprehensive simulation …
