Overdispersed count data are modelled with likelihood and non-likelihood approaches. Likelihood approaches include the Poisson mixtures with three distributions, the gamma, the lognormal, and the inverse Gaussian distributions. Non-likelihood approaches include the robust sandwich estimator and quasilikelihood. In this simulation study, overdispersed count data were simulated under the Poisson mixtures with the gamma, the lognormal and the inverse Gaussian distributions, then analyzed with the five likelihood and non-likelihood approaches. Our results indicated that 1) when the count data are mildly overdispersed, there are virtually no differences in type I error rate, standard error of the main effect, and empirical power among the five methods; 2) when the count data are very overdispersed, none of these five approaches is robust to model misspecification as evaluated by type I error rate, standard error of the main effect, and empirical power. This simulation study raises caution on using non-likelihood method for analyzing very overdispered count data because of likely higher type I error and inappropriate power levels. Unlike non-likelihood approaches, likelihood approaches allow for statistical tests based on likelihood ratios and for checking model fit and provide basis for power and sample size calculations. When likelihood approaches are used, we suggest comparing likelihood values to select the appropriate parametric method for analyzing very overdispersed count data.
Count data are ubiquitous in ecology and the Poisson generalized linear model (GLM) is commonly used to model the association between counts and explanatory variables of interest. When fitting this model to the data, one typically proceeds by first confirming that the data is not overdispersed and that there is no excess of zeros. If the data appear to be overdispersed or if there is any zero-inflation, key assumptions of the Poison GLM may be violated and researchers will then typically consider alternatives to the Poison GLM. An important question is whether the potential model selection bia…
We introduce a new class of Poisson-exponential-Tweedie (PET) mixture in the framework of generalized linear models for ultra-overdispersed count data. The mean-variance relationship is of the form $m+m^{2}+ϕm^{p}$, where $ϕ$ and $p$ are the dispersion and Tweedie power parameters, respectively. The proposed model is equivalent to the exponential-Poisson-Tweedie models arising from geometric sums of Poisson-Tweedie random variables. In this respect, the PET models encompass the geometric versions of Hermite, Neyman Type A, Pólya-Aeppli, negative binomial and Poisson inverse Gaussian models. Th…
This paper studies the non-asymptotic merits of the double $\ell_1$-regularized for heterogeneous overdispersed count data via negative binomial regressions. Under the restricted eigenvalue conditions, we prove the oracle inequalities for Lasso estimators of two partial regression coefficients for the first time, using concentration inequalities of empirical processes. Furthermore, derived from the oracle inequalities, the consistency and convergence rate for the estimators are the theoretical guarantees for further statistical inference. Finally, both simulations and a real data analysis demo…
We consider three new classes of exponential dispersion models of discrete probability distributions which are defined by specifying their variance functions in their mean value parameterization. In a previous paper (Bar-Lev and Ridder, 2020a), we have developed the framework of these classes and proved that they have some desirable properties. Each of these classes was shown to be overdispersed and zero inflated in ascending order, making them as competitive statistical models for those in use in statistical modeling. In this paper we elaborate on the computational aspects of their probabilit…
Generalized Poisson Matrix Factorization for Overdispersed Count Data
Graduate School of Science and Engineering, Chuo University（中央大学理工学研究科）；School of Pharmaceutical Sciences, Wakayama Medical University（和歌山医科大学药学部）；Faculty of Science and Engineering, Chuo University（中央大学理工学部）
Dynamic treatment regimens (DTRs), also known as treatment algorithms or adaptive interventions, play an increasingly important role in many health domains. DTRs are motivated to address the unique and changing needs of individuals by delivering the type of treatment needed, when needed, while minimizing unnecessary treatment. Practically, a DTR is a sequence of decision rules that specify, for each of several points in time, how available information about the individual's status and progress should be used in practice to decide which treatment (e.g., type or intensity) to deliver. The sequen…
Kolmogorov-Type Maximal Inequalities for Independent and Dependent Negative Binomial Random Variables: Sharp Bounds, Sub-Exponential Refinements, and Applications to Overdispersed Count Data
National Technical University of Athens（雅典国立技术大学）；Archimedes/Athena Research Center（阿尔基米德/雅典研究中心）；Canisius University（卡尼休斯大学）
Real count data time series often show the phenomenon of the underdispersion and overdispersion. In this paper, we develop two extensions of the first-order integer-valued autoregressive process with Poisson innovations, based on binomial thinning, for modeling integer-valued time series with equidispersion, underdispersion and overdispersion. The main properties of the models are derived. The methods of conditional maximum likelihood, Yule-Walker and conditional least squares are used for estimating the parameters, and their asymptotic properties are established. We also use a test based on o…
Negative binomial regression is commonly employed to analyze overdispersed count data. With small to moderate sample sizes, the maximum likelihood estimator of the dispersion parameter may be subject to a significant bias, that in turn affects inference on mean parameters. This paper proposes inference for negative binomial regression based on adjustments of the score function aimed at mean and median bias reduction. The resulting estimating equations are similar to those available for improved inference in generalized linear models and, in particular, can be solved using a suitable extension …
We present a new modelling approach for longitudinal count data that is motivated by the increasing availability of longitudinal RNA-sequencing experiments. The distribution of RNA-seq counts typically exhibits overdispersion, zero-inflation and heavy tails; moreover, in longitudinal designs repeated measurements from the same subject are typically (positively) correlated. We propose a generalized linear mixed model based on the Poisson-Tweedie distribution that can flexibly handle each of the aforementioned features of longitudinal overdispersed counts. We develop a computational approach to …
Conway-Maxwell-Poisson (CMP) distributions are flexible generalizations of the Poisson distribution for modelling overdispersed or underdispersed counts. The main hindrance to their wider use in practice seems to be the inability to directly model the mean of counts, making them not compatible with nor comparable to competing count regression models, such as the log-linear Poisson, negative-binomial or generalized Poisson regression models. This note illustrates how CMP distributions can be parametrized via the mean, so that simpler and more easily-interpretable mean-models can be used, such a…
Discrete distributions derived from renewal processes, ie distributions of the number of events by some time t are beginning to be used in econometrics and health sciences. A new fast method is presented for computation of the probabilities for these distributions. We calculate the count probabilities by repeatedly convolving the discretized distribution, and then correct them using Richardson extrapolation. When just one probability is required, a second algorithm is described, an adaptation of De Pril's method, in which the computation time does not depend on the ordinality, so that even hig…
Generalized linear latent variable models (GLLVMs) are a class of methods for analyzing multi-response data which has garnered considerable popularity in recent years, for example, in the analysis of multivariate abundance data in ecology. One of the main features of GLLVMs is their capacity to handle a variety of responses types, such as (overdispersed) counts, binomial responses, (semi-)continuous, and proportions data. On the other hand, the introduction of underlying latent variables presents some major computational challenges, as the resulting marginal likelihood function involves an int…
We propose a new class of discrete generalized linear models based on the class of Poisson-Tweedie factorial dispersion models with variance of the form $μ+ ϕμ^p$, where $μ$ is the mean, $ϕ$ and $p$ are the dispersion and Tweedie power parameters, respectively. The models are fitted by using an estimating function approach obtained by combining the quasi-score and Pearson estimating functions for estimation of the regression and dispersion parameters, respectively. This provides a flexible and efficient regression methodology for a comprehensive family of count models including Hermite, Neyman…
Univariate regression models have rich literature for counting data. However, this is not the case for multivariate count data. Therefore, we present the Multivariate Generalized Linear Mixed Models framework that deals with a multivariate set of responses, measuring the correlation between them through random effects that follows a multivariate normal distribution. This model is based on a GLMM with a random intercept and the estimation process remains the same as a standard GLMM with random effects integrated out via Laplace approximation. We efficiently implemented this model through the TM…
The premise of independence among subjects in the same cluster/group often fails in practice, and models that rely on such untenable assumption can produce misleading results. To overcome this severe deficiency, we introduce a new regression model to handle overdispersed and correlated clustered counts. To account for correlation within clusters, we propose a Poisson regression model where the observations within the same cluster are driven by the same latent random effect that follows the Birnbaum-Saunders distribution with a parameter that controls the strength of dependence among the indivi…
"For how many days during the past 30 days was your mental health not good?" The responses to this question measure self-reported mental health and can be linked to important covariates in the National Health and Nutrition Examination Survey (NHANES). However, these count variables present major distributional challenges: the data are overdispersed, zero-inflated, bounded by 30, and heaped in five- and seven-day increments. To meet these challenges, we design a semiparametric estimation and inference framework for count data regression. The data-generating process is defined by simultaneously …
This paper considers a general class of parameter-driven models for time series of counts. A comprehensive simulation study is conducted to evaluate the accuracy and efficiency of three estimators: the maximum likelihood estimators of the generalized linear model, 2-state finite mixture model, and 2-state hidden Markov model. Standard errors for these estimators are derived. Our results show that except in extreme cases, the maximum likelihood estimator of the generalized linear model is an efficient, consistent and robust estimator with a well-behaved estimated standard error. The maximum lik…
In the analysis of count data often the equidispersion assumption is not suitable, hence the Poisson regression model is inappropriate. As a generalization of the Poisson distribution, the COM-Poisson distribution can deal with under-, equi- and overdispersed count data. It is a member of the exponential family of distributions and has well known special cases. In spite of the nice properties of the COM-Poisson distribution, its location parameter does not correspond to the expectation, which complicates the interpretation of regression models. In this paper, we propose a straightforward repar…
When estimating a proportion and only a sample of triplets is given, dependencies within the triplets are to be accounted for. Without assuming a distribution for the success count of the triplet, together with the proportion, as second and third parameter the correlations of 1st and 2nd order enter the model. We apply maximum likelihood estimation, and derive consistency by using that the triplet count is multinomially distributed, combined with the continuous mapping theorem. The asymptotic normality follows with the delta-method, resulting in closed-form expressions for the standard errors.…
Count time series are widely encountered in practice. As with continuous valued data, many count series have seasonal properties. This paper uses a recent advance in stationary count time series to develop a general seasonal count time series modeling paradigm. The model permits any marginal distribution for the series and the most flexible autocorrelations possible, including those with negative dependence. Likelihood methods of inference can be conducted and covariates can be easily accommodated. The paper first develops the modeling methods, which entail a discrete transformation of a Gauss…
Unit-level models for survey data offer many advantages over their area-level counterparts, such as potential for more precise estimates and a natural benchmarking property. However two main challenges occur in this context: accounting for an informative survey design and handling non-Gaussian data types. The pseudo-likelihood approach is one solution to the former, and conjugate multivariate distribution theory offers a solution to the latter. By combining these approaches, we attain a unit-level model for count data that accounts for informative sampling designs and includes fully Bayesian m…
Latent autoregressive models are useful time series models for the analysis of infectious disease data. Evaluation of the likelihood function of latent autoregressive models is intractable and its approximation through simulation-based methods appears as a standard practice. Although simulation methods may make the inferential problem feasible, they are often computationally intensive and the quality of the numerical approximation may be difficult to assess. We consider instead a weighted pairwise likelihood approach and explore several computational and methodological aspects including estima…
In applications such as gene regulatory network analysis based on single-cell RNA sequencing data, samples often come from a mixture of different populations and each population has its own unique network. Available graphical models often assume that all samples are from the same population and share the same network. One has to first cluster the samples and use available methods to infer the network for every cluster separately. However, this two-step procedure ignores uncertainty in the clustering step and thus could lead to inaccurate network estimation. Motivated by these applications, we …
Inverse Probability Weighting of Count Exposures in the Presence of Missing Data: A Simulation Study
Centre for Longitudinal Studies, UCL Social Research Institute, University College London（长期研究发展中心，UCL社会研究所，伦敦大学学院）；Research Department of Behavioural Science and Health, Institute of Epidemiology and Health Care, University College London（行为科学与健康研究系，流行病学与健康护理研究所，伦敦大学学院）
In their fundamental paper on cubic variance functions, Letac and Mora (The Annals of Statistics,1990) presented a systematic, rigorous and comprehensive study of natural exponential families on the real line, their characterization through their variance functions and mean value parameterization. They presented a section that for some reason has been left unnoticed. This section deals with the construction of variance functions associated with natural exponential families of counting distributions on the set of nonnegative integers and allows to find the corresponding generating measures. As …
Federal University of São Carlos, University of São Paulo（巴西圣保罗联邦大学、圣保罗大学）；Institute of Mathematics and Computer Sciences, University of São Paulo（圣保罗大学数学与计算机科学学院）
[ABRIDGED] The Cash statistic, also known as the C stat, is commonly used for the analysis of low-count Poisson data, including data with null counts for certain values of the independent variable. The use of this statistic is especially attractive for low-count data that cannot be combined, or re-binned, without loss of resolution. This paper presents a new maximum-likelihood solution for the best-fit parameters of a linear model using the Poisson-based Cash statistic. The solution presented in this paper provides a new and simple method to measure the best-fit parameters of a linear model fo…
The spectrum of mutations in a collection of cancer genomes can be described by a mixture of a few mutational signatures. The mutational signatures can be found using non-negative matrix factorization (NMF). To extract the mutational signatures we have to assume a distribution for the observed mutational counts and a number of mutational signatures. In most applications, the mutational counts are assumed to be Poisson distributed, and the rank is chosen by comparing the fit of several models with the same underlying distribution and different values for the rank using classical model selection…
Assessing water quality and recognizing its associated risks to human health and the broader environment is undoubtedly essential. Groundwater is widely used to supply water for drinking, industry, and agriculture purposes. The groundwater quality measurements vary for different climates and various human behaviors, and consequently, their spatial variability can be substantial. In this paper, we aim to analyze a groundwater dataset from the Golestan province, Iran, for November 2003 to November 2013. Our target response variable to monitor the quality of groundwater is the number of counts th…
