This paper considers the issue of modeling fractional data observed in the interval [0,1), (0,1] or [0,1]. Mixed continuous-discrete distributions are proposed. The beta distribution is used to describe the continuous component of the model since its density can have quite diferent shapes depending on the values of the two parameters that index the distribution. Properties of the proposed distributions are examined. Also, maximum likelihood and method of moments estimation is discussed. Finally, practical applications that employ real data are presented.
In this article, we first introduced the inflated unit Lindley distribution considering zero or/and one inflation scenario and studied its basic distributional and structural properties. Both the distributions are shown to be members of exponential family with full rank. Different parameter estimation methods are discussed and supporting simulation studies to check their efficacy are also presented. Proportion of students passing the high school leaving examination for the schools across the state of Manipur in India for the year 2020 are then modeled using the proposed distributions and compa…
The inflated beta regression model aims to enable the modeling of responses in the intervals $(0,1]$, $[0,1)$ or $[0,1]$. In this model, hypothesis testing is often performed based on the likelihood ratio statistic. The critical values are obtained from asymptotic approximations, which may lead to distortions of size in small samples. In this sense, this paper proposes the bootstrap Bartlett correction to the statistic of likelihood ratio in the inflated beta regression model. The proposed adjustment only requires a simple Monte Carlo simulation. Through extensive Monte Carlo simulations the f…
This paper proposes a general class of regression models for continuous proportions when the data contain zeros or ones. The proposed class of models assumes that the response variable has a mixed continuous-discrete distribution with probability mass at zero or one. The beta distribution is used to describe the continuous component of the model, since its density has a wide range of different shapes depending on the values of the two parameters that index the distribution. We use a suitable parameterization of the beta law in terms of its mean and a precision parameter. The parameters of the …
A frequent challenge encountered with ecological data is how to interpret, analyze, or model data having a high proportion of zeros. Much attention has been given to zero-inflated count data, whereas models for non-negative continuous data with an abundance of 0s are lacking. We consider zero-inflated data on the unit interval and provide modeling to capture two types of 0s in the context of the Beta regression model. We model 0s due to missing by chance through left censoring of a latent regression, and 0s due to unsuitability using an independent Bernoulli specification to create a point mas…
Discrete ordinal responses such as Likert scales are regularly proposed in questionnaires and used as dependent variable in modeling. The response distribution for such scales is always discrete, with bounded support and often skewed. In addition, one particular level of the scale is frequently inflated as it cumulates respondents who invariably choose that particular level (typically the middle or one extreme of the scale) without hesitation with those who chose that alternative but might have selected a neighboring one. The inflated discrete beta regression (IDBR) model addresses those four …
In actuarial practice the dependency between contract limitations (deductibles, copayments) and health care expenditures are measured by the application of the Monte Carlo simulation technique. We propose, for the same goal, an alternative approach based on Generalized Linear Model for Location, Scale and Shape (GAMLSS). We focus on the estimate of the ratio between the one-year reimbursement amount (after the effect of limitations) and the one year expenditure (before the effect of limitations). We suggest a regressive model to investigate the relation between this response variable and a set…
Modeling fractional data in various real life scenarios is a challenging task. This paper consider situations where fractional data is observed on the interval [0,1]. The unit-Lindley distribution has been discussed in the literature where its support lies between 0 and 1. In this paper, we focus on an inflated variant of the unit-Lindley distribution, where the inflation occurs at both 0 and 1. Various properties of the inflated unit-Lindley distribution are discussed and examined, including point estimation based on the maximum likelihood method and interval estimation. Finally, extensive Mo…
Modeling sparse data such as microbiome and transcriptomics (RNA-seq) data is very challenging due to the exceeded number of zeros and skewness of the distribution. Many probabilistic models have been used for modeling sparse data, including Poisson, negative binomial, zero-inflated Poisson, and zero-inflated negative binomial models. One way to identify the most appropriate probabilistic models for zero-inflated or hurdle models is based on the p-value of the Kolmogorov-Smirnov (KS) test. The main challenge for identifying the probabilistic model is that the model parameters are typically unk…
This paper develops a Pareto scale-inflated outlier model. This model is intended for use when data from some standard Pareto distribution of interest is suspected to have been contaminated with a relatively small number of outliers from a Pareto distribution with the same shape parameter but with an inflated scale parameter. The Bayesian analysis of this Pareto scale-inflated outlier model is considered and its implementation using the Gibbs sampler is discussed. The paper contains three worked illustrative examples, two of which feature actual insurance claims data.
We consider modeling and prediction of Capelin distribution in the Barents sea based on zero-inflated count observation data that vary continuously over a specified survey region. The model is a mixture of two components; a one-point distribution at the origin and a Poisson distribution with spatio-temporal intensity, where both intensity and mixing proportions are modeled by some auxiliary variables and unobserved spatio-temporal effects. The spatio-temporal effects are modeled by a dynamic linear model combined with the predictive Gaussian process. We develop an efficient posterior computati…
This article introduces a k-Inflated Negative Binomial mixture distribution/regression model as a more flexible alternative to zero-inflated Poisson distribution/regression model. An EM algorithm has been employed to estimate the model's parameters. Then, such new model along with a Pareto mixture model have been employed to design an optimal rate--making system. Namely, this article employs number/size of reported claims of Iranian third party insurance dataset. Then, it employs the k-Inflated Negative Binomial mixture distribution/regression model as well as other well developed counting mod…
Item non-response in surveys is usually handled by single imputation, whose main objective is to reduce the non-response bias. Imputation methods need to be adapted to the study variable. For instance, in business surveys, the interest variables often contain a large number of zeros. Motivated by a mixture regression model, we propose two imputation procedures for such data and study their statistical properties. We show that these procedures preserve the distribution function if the imputation model is well specified. The results of a simulation study illustrate the good performance of the pr…
Researchers are often interested in predicting outcomes, conducting clustering analysis to detect distinct subgroups of their data, or computing causal treatment effects. Pathological data distributions that exhibit skewness and zero-inflation complicate these tasks - requiring highly flexible, data-adaptive modeling. In this paper, we present a fully nonparametric Bayesian generative model for continuous, zero-inflated outcomes that simultaneously predicts structural zeros, captures skewness, and clusters patients with similar joint data distributions. The flexibility of our approach yields p…
Models such as the zero-inflated and zero-altered Poisson and zero-truncated binomial are well-established in modern regression analysis. We propose a super model that jointly and maximally unifies alteration, inflation, truncation and deflation for counts, given a 1- or 2-parameter parent (base) distribution. Seven disjoint sets of special value types are accommodated because all but truncation have parametric and nonparametric variants. Some highlights include: (i) the mixture distribution is exceeding flexible, e.g., up to seven modes; (ii) under-, equi- and over-dispersion can be handled u…
Sequencing-based technologies provide an abundance of high-dimensional biological datasets with skewed and zero-inflated measurements. Classification of such data with linear discriminant analysis leads to poor performance due to the violation of the Gaussian distribution assumption. To address this limitation, we propose a new semiparametric discriminant analysis framework based on the truncated latent Gaussian copula model that accommodates both skewness and zero inflation. By applying sparsity regularization, we demonstrate that the proposed method leads to the consistent estimation of clas…
The main object of this article is to present an extension of the zero-inflated Poisson-Lindley distribution, called of zero-modified Poisson-Lindley. The additional parameter $π$ of the zero-modified Poisson-Lindley has a natural interpretation in terms of either zero-deflated/inflated proportion. Inference is dealt with by using the likelihood approach. In particular the maximum likelihood estimators of the distribution's parameter are compared in small and large samples. We also consider an alternative bias-correction mechanism based on Efron's bootstrap resampling. The model is applied to …
The human microbiome can contribute to pathogeneses of many complex diseases by mediating disease-leading causal pathways. However, standard mediation analysis methods are not adequate to analyze the microbiome as a mediator due to the excessive number of zero-valued sequencing reads in the data that is compounded by its compositional structure. The two main challenges raised by the zero-inflated data structure are: (a) disentangling the mediation effect induced by the point mass at zero; and (b) identifying the observed zero-valued data points that are actually not zero (i.e., false zeros). W…
We analyse tourism behaviour of Italian residents in the period covering the 2008 Great Recession. Using the Trips of Italian Residents in Italy and Abroad quarterly survey, carried out by the Italian National Institute of Statistics, we investigate whether and how the economic recession has affected the total number of overnight stays. The response variable is the result of a two-stage decision process: first we choose to take a holiday, then for how long. Moreover, since the number of overnight stays is typically concentrated on specific lengths (week-end, week, fortnight) we observe multipl…
Bulk gene expression experiments relied on aggregations of thousands of cells to measure the average expression in an organism. Advances in microfluidic and droplet sequencing now permit expression profiling in single cells. This study of cell-to-cell variation reveals that individual cells lack detectable expression of transcripts that appear abundant on a population level, giving rise to zero-inflated expression patterns. To infer gene co-regulatory networks from such data, we propose a multivariate Hurdle model. It is comprised of a mixture of singular Gaussian distributions. We employ neig…
Commonly in biomedical research, studies collect data in which an outcome measure contains informative excess zeros; for example when observing the burden of neuritic plaques in brain pathology studies, those who show none contribute to our understanding of neurodegenerative disease. The outcome may be characterized by a mixture distribution with one component being the `structural zero' and the other component being a Poisson distribution. We propose a novel variance components score test of genetic association between a set of genetic markers and a zero-inflated count outcome from a mixture …
Estimating the marginal and joint densities of the long-term average intakes of different dietary components is an important problem in nutritional epidemiology. Since these variables cannot be directly measured, data are usually collected in the form of 24-hour recalls of the intakes, which show marked patterns of conditional heteroscedasticity. Significantly compounding the challenges, the recalls for episodically consumed dietary components also include exact zeros. The problem of estimating the density of the latent long-time intakes from their observed measurement error contaminated proxi…
We consider three new classes of exponential dispersion models of discrete probability distributions which are defined by specifying their variance functions in their mean value parameterization. In a previous paper (Bar-Lev and Ridder, 2020a), we have developed the framework of these classes and proved that they have some desirable properties. Each of these classes was shown to be overdispersed and zero inflated in ascending order, making them as competitive statistical models for those in use in statistical modeling. In this paper we elaborate on the computational aspects of their probabilit…
The beta distribution is the best-known distribution for modelling doubly-bounded data, \eg percentage data or probabilities. A new generalization of the beta distribution is proposed, which uses a cubic transformation of the beta random variable. The new distribution is label-invariant like the beta distribution and has rational expressions for the moments. This facilitates its use in mean regression. The properties are discussed, and two examples of fitting to data are given. A modification is also explored in which the Jacobian of the transformation is omitted. This gives rise to messier ex…
This paper introduces the multivariate tail-inflated normal (MTIN) distribution, an elliptical heavy-tails generalization of the multivariate normal (MN). The MTIN belongs to the family of MN scale mixtures by choosing a convenient continuous uniform as mixing distribution. Moreover, it has a closed-form for the probability density function characterized by only one additional ``inflation'' parameter, with respect to the nested MN, governing the tail-weight. The first four moments are also computed; interestingly, they always exist and the excess kurtosis can assume any positive value. The met…
The most well known probability distribution of probabilities is the Beta distribution. If we have observed $r$ `successes', each having a probability $θ$, and $n-r$ `failures', each having a probability $1-θ$. In this paper we will derive a whole family of Beta-like distributions, which take as their data not only the number of successes and failures, but also values on predictor variables and time to failure or time without failure.
This paper proposes new linear regression models to deal with overdispersed binomial datasets. These new models, called tilted beta binomial regression models, are defined from the tilted beta binomial distribution, proposed assuming that the parameter of the binomial distribution follows a tilted beta distribution. As a particular case of this regression models, we propose the beta rectangular binomial regression models, defined from the binomial distribution assuming that their parameters follow a beta rectangular distribution. These new linear regression models, defined assuming that the pa…
Finite mixture representations of zero-and-$N$-inflated distributions for count-compositional data
Hamilton Institute and Department of Mathematics and Statistics, Maynooth University（梅努斯大学数学与统计系汉密尔顿研究所）；School of Mathematics and Statistics, Insight Centre for Data Analytics, University College Dublin（都柏林大学学院数据洞察中心数学与统计学院）
Tweedie's compound Poisson model is a popular method to model insurance claims with probability mass at zero and nonnegative, highly right-skewed distribution. In particular, it is not uncommon to have extremely unbalanced data with excessively large proportion of zero claims, and even traditional Tweedie model may not be satisfactory for fitting the data. In this paper, we propose a boosting-assisted zero-inflated Tweedie model, called EMTboost, that allows zero probability mass to exceed a traditional model. We makes a nonparametric assumption on its Tweedie model component, that unlike a li…
A marginalized three-part interrupted time series regression model for proportional data
Oregon Health & Science University（俄勒冈健康与科学大学）；Kaiser Permanente Washington Health Research Institute（凯撒永久华盛顿健康研究所）；University of Washington（华盛顿大学）；King’s College London（伦敦国王学院）
Modern RNA sequencing technologies provide gene expression measurements from single cells that promise refined insights on regulatory relationships among genes. Directed graphical models are well-suited to explore such (cause-effect) relationships. However, statistical analyses of single cell data are complicated by the fact that the data often show zero-inflated expression patterns. To address this challenge, we propose directed graphical models that are based on Hurdle conditional distributions parametrized in terms of polynomials in parent variables and their 0/1 indicators of being zero or…
The Dirichlet-multinomial (DM) distribution plays a fundamental role in modern statistical methodology development and application. Recently, the DM distribution and its variants have been used extensively to model multivariate count data generated by high-throughput sequencing technology in omics research due to its ability to accommodate the compositional structure of the data as well as overdispersion. A major limitation of the DM distribution is that it is unable to handle excess zeros typically found in practice which may bias inference. To fill this gap, we propose a novel Bayesian zero-…
Beta regression is often used to model the relationship between a dependent variable that assumes values on the open interval (0,1) and a set of predictor variables. An important challenge in beta regression is to find residuals whose distribution is well approximated by the standard normal distribution. Two previous works compared residuals in beta regression, but the authors did not include the quantile residual. Using Monte Carlo simulation techniques, this paper studies the behavior of certain residuals in beta regression in several scenarios. Overall, the results suggest that the distribu…
