We investigate stochastic comparisons between exponential family distributions and their mixtures with respect to the usual stochastic order, the hazard rate order, the reversed hazard rate order, and the likelihood ratio order. A general theorem based on the notion of relative log-concavity is shown to unify various specific results for the Poisson, binomial, negative binomial, and gamma distributions in recent literature. By expressing a convolution of gamma distributions with arbitrary scale and shape parameters as a scale mixture of gamma distributions, we obtain comparison theorems concerning such convolutions that generalize some known results. Analogous results on convolutions of negative binomial distributions are also discussed.
Recently Balakrishnan and Iliopoulos [Ann. Inst. Statist. Math. 61 (2009)] gave sufficient conditions under which maximum likelihood estimator (MLE) is stochastically increasing. In this paper we study test plans which are not considered there and we prove that MLEs for those plans are also stochastic ordered. We also give some applications to the estimation of reliability.
Recently, G.~Yanev obtained a characterization of the exponential family of distributions in terms of a functional equation for certain mixture densities. The purpose of this note is twofold: we extend Yanev's theorem by relaxing a restriction on the sign of mixture coefficients and, in addition, obtain a similar characterization for the Laplace family of distributions.
New results and improvements in the study of nonparametric exponential and mixture models are proposed. In particular, different equivalent characterizations of maximal exponential models, in terms of open exponential arcs and Orlicz spaces, are given. Our theoretical results are supported by several examples and counterexamples and provide an answer to some open questions in the literature.
We consider maximum likelihood estimation of finite mixture of uniform distributions. We prove that maximum likelihood estimator is strongly consistent, if the scale parameters of the component uniform distributions are restricted from below by exp(-n^d), 0 < d < 1, where n is the sample size.
We study the problem of finding the smallest $m$ such that every element of an exponential family can be written as a mixture of $m$ elements of another exponential family. We propose an approach based on coverings and packings of the face lattice of the corresponding convex support polytopes and results from coding theory. We show that $m=q^{N-1}$ is the smallest number for which any distribution of $N$ $q$-ary variables can be written as mixture of $m$ independent $q$-ary variables. Furthermore, we show that any distribution of $N$ binary variables is a mixture of $m = 2^{N-(k+1)}(1+ 1/(2^k-…
We investigate the geometry of the maximal a posteriori (MAP) partition in the Bayesian Mixture Model where the component and the base distributions are chosen from conjugate exponential families. We prove that in this case the clusters are separated by the contour lines of a linear functional of the sufficient statistic. As a particular example, we describe Bayesian Mixture of Normals with Normal-inverse-Wishart prior on the component mean and covariance, in which the clusters in any MAP partition are separated by a quadratic surface. In connection with results of Rajkowski (2018), where the …
Exponential families are the workhorses of parametric modelling theory. One reason for their popularity is their associated inference theory, which is very clean, both from a theoretical and a computational point of view. One way in which this set of tools can be enriched in a natural and interpretable way is through mixing. This paper develops and applies the idea of local mixture modelling to exponential families. It shows that the highly interpretable and flexible models which result have enough structure to retain the attractive inferential properties of exponential families. In particular…
In finite mixtures of location-scale distributions, if there is no constraint on the parameters then the maximum likelihood estimate does not exist. But when the ratios of the scale parameters are restricted appropriately, the maximum likelihood estimate exists. We prove that the maximum likelihood estimator (MLE) is strongly consistent, if the ratios of the scale parameters are restricted from below by $\exp(-n^{d}), 0 < d < 1 $, where $n$ is the sample size.
We show that the $k$th order statistic from a heterogeneous sample of $n\geq k$ exponential random variables is larger than that from a homogeneous exponential sample in the sense of star ordering, as conjectured by Xu and Balakrishnan (2012). As a consequence, we establish hazard rate ordering for order statistics between heterogeneous and homogeneous exponential samples, resolving an open problem of Pǎltǎnea (2008). Extensions to general spacings are also presented.
The normality assumption on data set is very restrictive approach for modelling. The generalized form of normal distribution, named as an exponential power (EP) distribution, and its scale mixture form have been considered extensively to overcome the problem for modelling non-normal data set since last decades. However, examining the robustness properties of maximum likelihood (ML) estimators of parameters in these distributions, such as the in uence function, gross-error sensitivity, breakdown point and information-standardized sensitivity, has not been considered together. The well-known asy…
In a finite mixture of location-scale distributions maximum likelihood estimator does not exist because of the unboundedness of the likelihood function when the scale parameter of some mixture component approaches zero. In order to study the strong consistency of maximum likelihood estimator, we consider the case that the scale parameters of the component distributions are restricted from below by $c_n$, where $c_n$ is a sequence of positive real numbers which tend to zero as the sample size $n$ increases. We prove that under mild regularity conditions maximum likelihood estimator is strongly …
Hazard rates play an important role in various areas, e.g., reliability theory, survival analysis, biostatistics, queueing theory and actuarial studies. Mixtures of distributions are also of a great preeminence in such areas as most populations of components are indeed heterogeneous. In this study we present a sufficient condition for mixtures of two elements\ of the same natural exponential family (NEF) to have an increasing hazard rate. We then apply this condition to some classical NEF's having either quadratic, or cubic variance functions (VF) and others as well. A particular attention is …
We study the behavior of exponential random graphs in both the sparse and the dense regime. We show that exponential random graphs are approximate mixtures of graphs with independent edges whose probability matrices are critical points of an associated functional, thereby satisfying a certain matrix equation. In the dense regime, every solution to this equation is close to a block matrix, concluding that the exponential random graph behaves roughly like a mixture of stochastic block models. We also show existence and uniqueness of solutions to this equation for several families of exponential …
Let $X_1, X_2,\ldots, X_n$ (resp. $Y_1, Y_2,\ldots, Y_n$) be independent random variables such that $X_i$ (resp. $Y_i$) follows generalized exponential distribution with shape parameter $θ_i$ and scale parameter $λ_i$ (resp. $δ_i$), $i=1,2,\ldots, n$. Here it is shown that if $\left(λ_1, λ_2,\ldots,λ_n\right)$ is $p$-larger than (resp. weakly supermajorizes) $\left(δ_1,δ_2,\ldots,δ_n\right)$, then $X_{n:n}$ will be greater than $Y_{n:n}$ in usual stochastic order (resp. reversed hazard rate order). That no relation exists between $X_{n:n}$ and $Y_{n:n}$, under same condition, in terms of likel…
In this paper we are interested in testing whether there are any signals hidden in high dimensional noise data. Therefore we study the family of goodness-of-fit tests based on $Φ$-divergences including the test of Berk and Jones as well as Tukey's higher criticism test. The optimality of this family is already known for the heterogeneous normal mixture model. We now present a technique to transfer this optimality to more general models. For illustration we apply our results to dense signal and sparse signal models including the exponential-$χ^2$ mixture model and general exponential families a…
We consider the learning task consisting in predicting as well as the best function in a finite reference set G up to the smallest possible additive term. If R(g) denotes the generalization error of a prediction function g, under reasonable assumptions on the loss function (typically satisfied by the least square loss when the output is bounded), it is known that the progressive mixture rule g_n satisfies E R(g_n) < min_{g in G} R(g) + C (log|G|)/n where n denotes the size of the training set, E denotes the expectation w.r.t. the training set distribution and C denotes a positive constant. Thi…
Sparse models are desirable for many applications across diverse domains as they can perform automatic variable selection, aid interpretability, and provide regularization. When fitting sparse models in a Bayesian framework, however, analytically obtaining a posterior distribution over the parameters of interest is intractable for all but the simplest cases. As a result practitioners must rely on either sampling algorithms such as Markov chain Monte Carlo or variational methods to obtain an approximate posterior. Mean field variational inference is a particularly simple and popular framework t…
For two vast families of mixture distributions and a given prior, we provide unified representations of posterior and predictive distributions. Model applications presented include bivariate mixtures of Gamma distributions labelled as Kibble-type, non-central Chi-square and F distributions, the distribution of $R^2$ in multiple regression, variance mixture of normal distributions, and mixtures of location-scale exponential distributions including the multivariate Lomax distribution. An emphasis is also placed on analytical representations and the relationships with a host of existing distribut…
This paper considers a family of distributions constructed by a stochastic mixture of the order statistics of a sample of size two. Various properties of the proposed model are studied. We apply the model to extend the exponential and symmetric Laplace distributions. An extension to the bivariate case is considered.
We establish exponential bounds for the hypergeometric distribution which include a finite sampling correction factor, but are otherwise analogous to bounds for the binomial distribution due to León and Perron (2003) and Talagrand (1994). We also establish a convex ordering for sampling without replacement from populations of real numbers between zero and one: a population of all zeros or ones (and hence yielding a hypergeometric distribution in the upper bound) gives the extreme case.
Discrete mixture models provide a well-known basis for effective clustering algorithms, although technical challenges have limited their scope. In the context of gene-expression data analysis, a model is presented that mixes over a finite catalog of structures, each one representing equality and inequality constraints among latent expected values. Computations depend on the probability that independent gamma-distributed variables attain each of their possible orderings. Each ordering event is equivalent to an event in independent negative-binomial random variables, and this finding guides a dy…
