In this short note, we prove an asymptotic expansion for the ratio of the Dirichlet density to the multivariate normal density with the same mean and covariance matrix. The expansion is then used to derive an upper bound on the total variation between the corresponding probability measures and rederive the asymptotic variance of the Dirichlet kernel estimators introduced by Aitchison & Lauder (1985) and studied theoretically in Ouimet (2020). Another potential application related to the asymptotic equivalence between the Gaussian variance regression problem and the Gaussian white noise problem is briefly mentioned but left open for future research.
In this paper, the maximum spacing method is considered for multivariate observations. Nearest neighbour balls are used as a multidimensional analogue to univariate spacings. A class of information-type measures is used to generalize the concept of maximum spacing estimators. Asymptotic normality of these generalized maximum spacing estimators is proved when the assigned model class is correct, that is the true density is a member of the model class.
The asymptotic normality of the maximum likelihood estimator (MLE) under regularity conditions is a cornerstone of statistical theory. In this paper, we give explicit upper bounds on the distributional distance between the distribution of the MLE of a vector parameter, and the multivariate normal distribution. We work with possibly high-dimensional, independent but not necessarily identically distributed random vectors. In addition, we obtain explicit upper bounds even in cases where the MLE cannot be expressed analytically.
We use the delta method and Stein's method to derive, under regularity conditions, explicit upper bounds for the distributional distance between the distribution of the maximum likelihood estimator (MLE) of a $d$-dimensional parameter and its asymptotic multivariate normal distribution. Our bounds apply in situations in which the MLE can be written as a function of a sum of i.i.d. $t$-dimensional random vectors. We apply our general bound to establish a bound for the multivariate normal approximation of the MLE of the normal distribution with unknown mean and variance.
The problem of characterizing a multivariate distribution of a random vector using examination of univariate combinations of vector components is an essential issue of multivariate analysis. The likelihood principle plays a prominent role in developing powerful statistical inference tools. In this context, we raise the question: can the univariate likelihood function based on a random vector be used to provide the uniqueness in reconstructing the vector distribution? In multivariate normal (MN) frameworks, this question links to a reverse of Cochran's theorem that concerns the distribution of …
We obtain explicit $p$-Wasserstein distance error bounds between the distribution of the multi-parameter MLE and the multivariate normal distribution. Our general bounds are given for possibly high-dimensional, independent and identically distributed random vectors. Our general bounds are of the optimal $\mathcal{O}(n^{-1/2})$ order. Explicit numerical constants are given when $p\in(1,2]$, and in the case $p>2$ the bounds are explicit up to a constant factor that only depends on $p$. We apply our general bounds to derive Wasserstein distance error bounds for the multivariate normal approximati…
In this work, the estimation of the multivariate normal mean by different classes of shrinkage estimators is investigated. The risk associated with the balanced loss function is used to compare two estimators. We start by considering estimators that generalize the James-Stein estimator and show that these estimators dominate the maximum likelihood estimator (MLE), therefore are minimax, when the shrinkage function satisfies some conditions. Then, we treat estimators of polynomial form and prove the increase of the degree of the polynomial allows us to build a better estimator from the one prev…
Multivariate processes with long-range dependence properties can be encountered in many fields of application. Two fundamental characteristics in such frameworks are long-range dependence parameters and correlations between component time series. We consider multivariate long-range dependent linear processes, not necessarily Gaussian. We show that the covariances between the wavelet coefficients in this setting are asymptotically Gaussian. We also study the asymptotic distributions of the estimators of the long-range dependence parameter and the long-run covariance by a wavelet-based Whittle p…
Semiparametric estimation for multivariate Hawkes processes using dependent Dirichlet processes: An application to order flow data in financial markets
We study a novel class of affine invariant and consistent tests for multivariate normality. The tests are based on a characterization of the standard $d$-variate normal distribution by means of the unique solution of an initial value problem connected to a partial differential equation, which is motivated by a multivariate Stein equation. The test criterion is a suitably weighted $L^2$-statistic. We derive the limit distribution of the test statistic under the null hypothesis as well as under contiguous and fixed alternatives to normality. A consistent estimator of the limiting variance under …
In this paper, a new mixture family of multivariate normal distributions, formed by mixing multivariate normal distribution and skewed distribution, is constructed. Some properties of this family, such as characteristic function, moment generating function, and the first four moments are derived. The distributions of affine transformations and canonical forms of the model are also derived. An EM type algorithm is developed for the maximum likelihood estimation of model parameters. We have considered in detail, some special cases of the family, using standard gamma and standard exponential mixt…
Multivariate normal mixtures provide a flexible model for high-dimensional data. They are widely used in statistical genetics, statistical finance, and other disciplines. Due to the unboundedness of the likelihood function, classical likelihood-based methods, which may have nice practical properties, are inconsistent. In this paper, we recommend a penalized likelihood method for estimating the mixing distribution. We show that the maximum penalized likelihood estimator is strongly consistent when the number of components has a known upper bound. We also explore a convenient EM-algorithm for co…
We study a novel class of affine invariant and consistent tests for normality in any dimension. The tests are based on a characterization of the standard $d$-variate normal distribution as the unique solution of an initial value problem of a partial differential equation motivated by the harmonic oscillator, which is a special case of a Schrödinger operator. We derive the asymptotic distribution of the test statistics under the hypothesis of normality as well as under fixed and contiguous alternatives. The tests are consistent against general alternatives, exhibit strong power performance for …
The Dirichlet distribution, also known as multivariate beta, is the most used to analyse frequencies or proportions data. Maximum likelihood is widespread for estimation of Dirichlet's parameters. However, for small sample sizes, the maximum likelihood estimator may shows a significant bias. In this paper, Dirchlet's parameters estimation is obtained through modified score functions aiming at mean and median bias reduction of the maximum likelihood estimator, respectively. A simulation study and an application compare the adjusted score approaches with maximum likelihood.
The skew-normal and related families are flexible and asymmetric parametric models suitable for modelling a diverse range of systems. We show that the multivariate maximum of a high-dimensional extended skew-normal random sample has asymptotically independent components and derive the speed of convergence of the joint tail. To describe the possible dependence among the components of the multivariate maximum, we show that under appropriate conditions an approximate multivariate extreme-value distribution that leads to a rich dependence structure can be derived.
We revisit multivariate extreme value theory modeling by emphasizing multivariate regular variations and the multivariate Breiman Lemma. This allows us to recover in a simple framework the most popular multivariate extreme value distributions, such as the logistic, negative logistic, Dirichlet, extremal-$t$ and Hüsler-Reiss models. In a second part of the paper, we focus on the Hüsler-Reiss Pareto model and its surprising exponential family property. After a thorough study of this exponential family structure, we focus on maximum likelihood estimation. We also consider the generalized Hüsler-R…
Azzalini (1985) introduced a skew-normal distribution of which normal distribution is a special case. Recently Kundu (2014) introduced a geometric skew-normal distribution and showed that it has certain advantages over Azzalini's skew-normal distribution. In this paper we discuss about the multivariate geometric skew-normal distribution. It can be used as an alternative to Azzalini's skew normal distribution. We discuss different properties of the proposed distribution. It is observed that the joint probability density function of the multivariate geometric skew normal distribution can take va…
Most normality tests in the literature are performed for scalar and independent samples. Thus, they become unreliable when applied to colored processes, hampering their use in realistic scenarios.We focus on Mardia's multivariate kurtosis, derive closed-form expressions of its asymptotic distribution for statistically dependent samples, under the null hypothesis of normality and a mixing condition. The calculus is long and tedious but the final result is simple and is implemented with a low computational burden. The proposed expression of the test power exhibits good properties on various scen…
Handling Missingness and Censoring in Dirichlet Models
University of Pretoria（比勒陀利亚大学）；National Institute for Theoretical and Computational Sciences (NITheCS)（国家理论与计算科学研究所）；San José State University（圣何塞州立大学）；University of Catania（卡塔尼亚大学）
It is shown that a simple Dirichlet process mixture of multivariate normals offers Bayesian density estimation with adaptive posterior convergence rates. Toward this, a novel sieve for non-parametric mixture densities is explored, and its rate adaptability to various smoothness classes of densities in arbitrary dimension is demonstrated. This sieve construction is expected to offer a substantial technical advancement in studying Bayesian non-parametric mixture models based on stick-breaking priors.
We show that rate-adaptive multivariate density estimation can be performed using Bayesian methods based on Dirichlet mixtures of normal kernels with a prior distribution on the kernel's covariance matrix parameter. We derive sufficient conditions on the prior specification that guarantee convergence to a true density at a rate that is optimal minimax for the smoothness class to which the true density belongs. No prior knowledge of smoothness is assumed. The sufficient conditions are shown to hold for the Dirichlet location mixture of normals prior with a Gaussian base measure and an inverse-W…
Our article addresses the problem of flexibly estimating a multivariate density while also attempting to estimate its marginals correctly. We do so by proposing two new estimators that try to capture the best features of mixture of normals and copula estimators while avoiding some of their weaknesses. The first estimator we propose is a mixture of normals copula model that is a flexible alternative to parametric copula models such as the normal and t copula. The second is a marginally adapted mixture of normals estimator that improves on the standard mixture of normals by using information con…
The mixture of factor analyzers (MFA) model provides a powerful tool for analyzing high-dimensional data as it can reduce the number of free parameters through its factor-analytic representation of the component covariance matrices. This paper extends the MFA model to incorporate a restricted version of the multivariate skew-normal distribution to model the distribution of the latent component factors, called mixtures of skew-normal factor analyzers (MSNFA). The proposed MSNFA model allows us to relax the need for the normality assumption for the latent factors in order to accommodate skewness…
There exist several endeavors proposing a new family of extended distributions using the beta-generating technique. This is a well-known mechanism in developing flexible distributions, by embedding the cumulative distribution function (cdf) of a baseline distribution within the beta distribution that acts as a generator. Univariate beta-generated distributions offer many fruitful and tractable properties and have applications in hydrology, biology and environmental sciences amongst other fields. In the univariate cases, this extension works well, however, for multivariate cases, the beta distr…
Gibbs partition models are the largest class of infinite exchangeable partitions of the positive integers generalizing the product form of the probability function of the two-parameter Poisson-Dirichlet family. Recently those models have been investigated in a Bayesian nonparametric approach to species sampling problems as alternatives to the Dirichlet and the Pitman-Yor process priors. Here we derive marginals of conditional and unconditional multivariate distributions arising from exchangeable Gibbs partitions to obtain explicit formulas for joint falling factorial moments of corresponding c…
In this short note, we develop a local approximation for the log-ratio of the multivariate hypergeometric probability mass function over the corresponding multinomial probability mass function. In conjunction with the bounds from Carter (2002) and Ouimet (2021) on the total variation between the law of a multinomial vector jittered by a uniform on $(-1/2,1/2)^d$ and the law of the corresponding multivariate normal distribution, the local expansion for the log-ratio is then used to obtain a total variation bound between the law of a multivariate hypergeometric random vector jittered by a unifor…
In this paper, a novel Bayesian nonparametric test for assessing multivariate normal models is presented. While there are extensive frequentist and graphical methods for testing multivariate normality, it is challenging to find Bayesian counterparts. The proposed approach is based on the use of the Dirichlet process and Mahalanobis distance. More precisely, the Mahalanobis distance is employed as a good technique to transform the $m$-variate problem into a univariate problem. Then the Dirichlet process is used as a prior on the distribution of the Mahalanobis distance. The concentration of the…
Stochastic linear combinations of some random vectors are studied where the distribution of the random vectors and the joint distribution of their coefficients are Dirichlet. A method is provided for calculating the distribution of these combinations which has been studied before by some authors. Our main result is a generalization of some existing results with a simpler proof.
In his 1986 book, Aitchison explains that compositional data is regularly mishandled in statistical analyses, a pattern that continues to this day. The Dirichlet Type I distribution is a multivariate distribution commonly used to model a set of proportions that sum to one. Aitchinson goes on to lament the difficulties of Dirichlet modelling and the scarcity of alternatives. While he addresses the second of these issues, we address the first. The Dirichlet Type II distribution is a transformation of the Dirichlet Type I distribution and is a multivariate distribution on the positive real number…
Density estimation represents one of the most successful applications of Bayesian nonparametrics. In particular, Dirichlet process mixtures of normals are the gold standard for density estimation and their asymptotic properties have been studied extensively, especially in the univariate case. However a gap between practitioners and the current theoretical literature is present. So far, posterior asymptotic results in the multivariate case are available only for location mixtures of Gaussian kernels with independent prior on the common covariance matrix, while in practice as well as from a conc…
An admissible estimator of the eigenvalues of the variance-covariance matrix is given for multivariate normal distributions with respect to the scale-invariant squared error loss.
We study admissibility of a subclass of generalized Bayes estimators of a multivariate normal vector when the variance is unknown, under scaled quadratic loss. Minimaxity is also established for certain of these estimators.
Several formulations have long existed in the literature in the form of continuous mixtures of normal variables where a mixing variable operates on the mean or on the variance or on both the mean and the variance of a multivariate normal variable, by changing the nature of these basic constituents from constants to random quantities. More recently, other mixture-type constructions have been introduced, where the core random component, on which the mixing operation operates, is not necessarily normal. The main aim of the present work is to show that many existing constructions can be encompasse…
This paper develops recurrence relations for integrals that relate the density of multivariate extended skew-normal (ESN) distribution, including the well-known skew-normal (SN) distribution introduced by Azzalini and Dalla-Valle (1996) and the popular multivariate normal distribution. These recursions offer a fast computation of arbitrary order product moments of the multivariate truncated extended skew-normal and multivariate folded extended skew-normal distributions with the product moments as a byproduct. In addition to the recurrence approach, we realized that any arbitrary moment of the …
This short note considers the problem of testing the null hypothesis that the mean values of two multivariate normal variables are proportional. We show that the usual likelihood ratio $χ^2$-test is valid non-asymptotically. Our proof relies on expressing the test statistic as the minimum eigenvalue of a Wishart variable and using a representation of its distribution using Legendre polynomials.
The problem of estimating a mean matrix of a multivariate complex normal distribution with an unknown covariance matrix is considered under an invariant loss function. By using complex versions of the Stein identity, the Stein-Haff identity, and calculus on eigenvalues, a formula is obtained for an unbiased estimate of the risk of an invariant class of estimators, from which several minimax shrinkage estimators are constructed.
We show that the orthogonal projection operator onto the range of the adjoint of a linear operator T can be represented as UT, where U is an invertible linear operator. Using this representation we obtain a decomposition of a multivariate Normal random variable Y as the sum of a linear transformation of Y that is independent of TY and an affine transformation of TY. We then use this decomposition to prove that the regular conditional distribution of a multivariate Normal random variable Y given a linear transformation TY is again a multivariate Normal distribution. This result is equivalent to…
We consider the problem of testing multivariate normality when the data consists of a random sample of two-step monotone incomplete observations. We define for such data a generalization of Mardia's statistic for measuring kurtosis, derive the asymptotic non-null distribution of the statistic under certain regularity conditions and against a broad class of alternatives, and give an application to a well-known data set on cholesterol measurements.
We consider an inference on the eigenvalues of the covariance matrix of a multivariate normal distribution. The family of multivariate normal distributions with a fixed mean is seen as a Riemannian manifold with Fisher information metric. Two submanifolds naturally arises; one is the submanifold given by fixed eigenvectors of the covariance matrix, the other is the one given by fixed eigenvalues. We analyze the geometrical structures of these manifolds such as metric, embedding curvature under $e$-connection or $m$-connection. Based on these results, we study 1) the bias of the sample eigenval…
The distance and divergence of the probability measures play a central role in statistics, machine learning, and many other related fields. The Wasserstein distance has received much attention in recent years because of its distinctions from other distances or divergences. Although~computing the Wasserstein distance is costly, entropy-regularized optimal transport was proposed to computationally efficiently approximate the Wasserstein distance. The purpose of this study is to understand the theoretical aspect of entropy-regularized optimal transport. In this paper, we~focus on entropy-regulari…
In this article we provide some nonnegative and positive estimators of the mean squared errors(MSEs) for shrinkage estimators of multivariate normal means. Proposed estimators are shown to improve on the uniformly minimum variance unbiased estimator(UMVUE) under a quadratic loss criterion. A similar improvement is also obtained for the estimators of the MSE matrices for shrinkage estimators. We also apply the proposed estimators of the MSE matrix to form confidence sets centered at shrinkage estimators and show their usefulness through numerical experiments.
We consider the problem of estimating covariance and precision matrices, and their associated discriminant coefficients, from normal data when the rank of the covariance matrix is strictly smaller than its dimension and the available sample size. Using unbiased risk estimation, we construct novel estimators by minimizing upper bounds on the difference in risk over several classes. Our proposal estimates are empirically demonstrated to offer substantial improvement over classical approaches.
This paper mainly focuses on studying the Shannon Entropy and Kullback-Leibler divergence of the multivariate log canonical fundamental skew-normal (LCFUSN) and canonical fundamental skew-normal (CFUSN) families of distributions, extending previous works. We relate our results with other well known distributions entropies. As a byproduct, we also obtain the Mutual Information for distributions in these families. Shannon entropy is used to compare models fitted to analyze the USA monthly precipitation data. Kullback-Leibler divergence is used to cluster regions in Atlantic ocean according to th…
Statistical depths have been well studied for multivariate and functional data over the past few decades, but remain under-explored for point processes. A first attempt on the notion of point process depth was conducted recently where the depth was defined as a weighted product of two terms: (1) the probability of the number of events in each process and (2) the depth of the event times conditioned on the number of events by using a Mahalanobis depth. We point out that multivariate depths such as the Mahalanobis depth cannot be directly used because they often neglect the important ordered pro…
Robust estimation under multivariate normal (MVN) mixture model is always a computational challenge. A recently proposed maximum pseudo \b{eta}-likelihood estimator aims to estimate the unknown parameters of a MVN mixture model in the spirit of minimum density power divergence (DPD) methodology but with a relatively simpler and tractable computational algorithm even for larger dimensions. In this letter, we will rigorously derive the existence and weak consistency of the maximum pseudo \b{eta}-likelihood estimator in case of MVN mixture models under a reasonable set of assumptions.
The normalized maximum likelihood (NML) is one of the most important distribution in coding theory and statistics. NML is the unique solution (if exists) to the pointwise minimax regret problem. However, NML is not defined even for simple family of distributions such as the normal distributions. Since there does not exist any meaningful minimax-regret distribution for such case, it is pointed out that NML with luckiness (LNML) can be employed as an alternative to NML. In this paper, we develop the closed form of LNMLs for multivariate normal distributions.
In this article, we consider two forms of shrinkage estimators of the mean $θ$ of a multivariate normal distribution $X\sim N_{p}\left(θ, σ^{2}I_{p}\right)$ where $σ^{2}$ is unknown. We take the prior law $θ\sim N_{p}\left(\upsilon, τ^{2}I_{p}\right)$ and we constuct a Modified Bayes estimator $δ_{B}^{\ast}$ and an Empirical Modified Bayes estimator $δ_{EB}^{\ast}$. We are interested in studying the minimaxity and the limits of risks ratios of these estimators, to the maximum likelihood estimator $X$, when $n$ and $p$ tend to infinity.
We show that the orthogonal projection operator onto the range of the adjoint of a linear operator $T$ can be represented as $UT,$ where $U$ is an invertible linear operator. Using this representation we obtain a decomposition of a Normal random vector $Y$ as the sum of a linear transformation of $Y$ that is independent of $TY$ and an affine transformation of $TY$. We then use this decomposition to prove that the conditional distribution of a Normal random vector $Y$ given a linear transformation $\mathcal{T}Y$ is again a multivariate Normal distribution. This result is equivalent to the well-…
We use the holonomic gradient method to evaluate the probability content of a simplex region under a multivariate normal distribution. This probability equals to the integral of the probability density function of the multivariate Gaussian distribution on the simplex region. For this purpose, we generalize the inclusion--exclusion identity which was given for polyhedra, to the faces of a polyhedron. This extended inclusion--exclusion identity enables us to calculate the derivatives of the function associated with the probability content of a polyhedron in general position. We show that these d…
Finite mixtures of multivariate normal distributions have been widely used in empirical applications in diverse fields such as statistical genetics and statistical finance. Testing the number of components in multivariate normal mixture models is a long-standing challenge even in the most important case of testing homogeneity. This paper develops likelihood-based tests of the null hypothesis of $M_0$ components against the alternative hypothesis of $M_0 + 1$ components for a general $M_0 \geq 1$. For heteroscedastic normal mixtures, we propose an EM test and derive the asymptotic distribution …
We show that lower-dimensional marginal densities of dependent zero-mean normal distributions truncated to the positive orthant exhibit a mass-shifting phenomenon. Despite the truncated multivariate normal density having a mode at the origin, the marginal density assigns increasingly small mass near the origin as the dimension increases. The phenomenon accentuates with stronger correlation between the random variables. A precise quantification characterizing the role of the dimension as well as the dependence is provided. This surprising behavior has serious implications towards Bayesian const…
We investigate predictive densities for multivariate normal models with unknown mean vectors and known covariance matrices. Bayesian predictive densities based on shrinkage priors often have complex representations, although they are effective in various problems. We consider extended normal models with mean vectors and covariance matrices as parameters, and adopt predictive densities that belong to the extended models including the original normal model. We adopt predictive densities that are optimal with respect to the posterior Bayes risk in the extended models. The proposed predictive dens…
Multivariate normal mixtures provide a flexible method of fitting high-dimensional data. It is shown that their topography, in the sense of their key features as a density, can be analyzed rigorously in lower dimensions by use of a ridgeline manifold that contains all critical points, as well as the ridges of the density. A plot of the elevations on the ridgeline shows the key features of the mixed density. In addition, by use of the ridgeline, we uncover a function that determines the number of modes of the mixed density when there are two components being mixed. A followup analysis then give…
