The paper aims at finding widely and smoothly defined nonparametric location and scatter functionals. As a convenient vehicle, maximum likelihood estimation of the location vector m and scatter matrix S of an elliptically symmetric t distribution on d-dimensional space with degrees of freedom larger than 1 extends to an M-functional defined on all probability distributions P in a weakly open, weakly dense domain U. Here U consists of P not putting too much mass in hyperplanes of dimension < d, as shown for empirical measures by Kent and Tyler, Ann. Statist. 1991. It is shown here that (m,S) is analytic on U, for the bounded Lipschitz norm, or for d=1, for the sup norm on distribution functions. For k=1,2,..., and other norms, depending on k and more directly adapted to t functionals, one has continuous differentiability of order k, allowing the delta-method to be applied to (m,S) for any P in U, which can be arbitrarily heavy-tailed. These results imply asymptotic normality of the corresponding M-estimators (m_n,S_n). In dimension d=1 only, the t functionals extend to be defined and weakly continuous at all t.
This paper proposes a local representation for Empirical Likelihood (EL). EL admits the classical local linear quadratic representation by its likelihood ratio property. A local estimator is derived by using the new representation. Consistency, local asymptotic normality, and asymptotic optimality results hold for the new estimator. In particular, when the regularity conditions do not include any differentiability assumption, these asymptotic results are still valid for the local estimator. Simulations illustrate that the local method improves the inference accuracy of EL.
The vanilla method in univariate extreme-value theory consists of fitting the three-parameter Generalized Extreme-Value (GEV) distribution to a sample of block maxima. Despite claims to the contrary, the asymptotic normality of the maximum likelihood estimator has never been established. In this paper, a formal proof is given using a general result on the maximum likelihood estimator for parametric families that are differentiable in quadratic mean but whose supports depend on the parameter. An interesting side result concerns the (lack of) differentiability in quadratic mean of the GEV family…
In this paper, we study the differentiability of implicitly defined functions which we encounter in the profile likelihood estimation of parameters in semi-parametric models. Scott and Wild (Biometrika 84 (1997) 57-71; J. Statist. Plann. Inference 96 (2001) 3-27) and Murphy and van der Vaart (J. Amer. Statist. Assoc. 95 (2000) 449-485) developed methodologies that can avoid dealing with such implicitly defined functions by parametrizing parameters in the profile likelihood and using an approximate least favorable submodel in semi-parametric models. Our result shows applicability of an alternat…
A generalized linear point process is specified in terms of an intensity that depends upon a linear predictor process through a fixed non-linear function. We present a framework where the linear predictor is parametrized by a Banach space and give results on Gateaux differentiability of the log-likelihood. Of particular interest is when the intensity is expressed in terms of a linear filter parametrized by a Sobolev space. Using that the Sobolev spaces are reproducing kernel Hilbert spaces we derive results on the representation of the penalized maximum likelihood estimator in a special case a…
In proofs of L_2-differentiability, Lebesgue densities of a central distribution are often assumed right from the beginning. Generalizing Theorem 4.2 of Huber[81], we show that in the class of smooth parametric group models these densities are in fact consequences of a finite Fisher information of the model, provided a suitable representation of the latter is used. The proof uses the notions of absolute continuity in k dimensions and weak differentiability. As examples to which this theorem applies, we spell out a number of models including a correlation model and the general multivariate loca…
We relate scattering amplitudes in particle physics to maximum likelihood estimation for discrete models in algebraic statistics. The scattering potential plays the role of the log-likelihood function, and its critical points are solutions to rational function equations. We study the ML degree of low-rank tensor models in statistics, and we revisit physical theories proposed by Arkani-Hamed, Cachazo and their collaborators. Recent advances in numerical algebraic geometry are employed to compute and certify critical points. We also discuss positive models and how to compute their string amplitu…
Suppose that $\m{U} = (U_1, \ldots , U_d) $ has a Uniform$([0,1]^d)$ distribution, that $\m{Y} = (Y_1 , \ldots , Y_d) $ has the distribution $G$ on $\RR_+^d$, and let $\m{X} = (X_1 , \ldots , X_d) = (U_1 Y_1 , \ldots , U_d Y_d )$. The resulting class of distributions of $\m{X}$ (as $G$ varies over all distributions on $\RR_+^d$) is called the {\sl Scale Mixture of Uniforms} class of distributions, and the corresponding class of densities on $\RR_+^d$ is denoted by $\{\cal F}_{SMU}(d)$. We study maximum likelihood estimation in the family ${\cal F}_{SMU}(d)$. We prove existence of the MLE, esta…
A novel approach to obtain weighted likelihood estimates of multivariate location and scatter is discussed. A weighting scheme is proposed that is based on the distribution of the Mahalanobis distances rather than the distribution of the data at the assumed model. This strategy allows to avoid the curse of dimensionality affecting non-parametric density estimation, that is involved in the construction of the weights through the Pearson residuals Markatou et al (1998). Then, weighted likelihood based outlier detection rules and robust dimensionality reduction techniques are developed. The effec…
Nonparametric Density Estimation for Data Scattered on Irregular Spatial Domains: A Likelihood-Based Approach Using Bivariate Penalized Spline Smoothing
We propose a class of locally and asymptotically optimal tests, based on multivariate ranks and signs for the homogeneity of scatter matrices in $m$ elliptical populations. Contrary to the existing parametric procedures, these tests remain valid without any moment assumptions, and thus are perfectly robust against heavy-tailed distributions (validity robustness). Nevertheless, they reach semiparametric efficiency bounds at correctly specified elliptical densities and maintain high powers under all (efficiency robustness). In particular, their normal-score version outperforms traditional Gaussi…
Despite the ubiquity of the Gaussian process regression model, few theoretical results are available that account for the fact that parameters of the covariance kernel typically need to be estimated from the dataset. This article provides one of the first theoretical analyses in the context of Gaussian process regression with a noiseless dataset. Specifically, we consider the scenario where the scale parameter of a Sobolev kernel (such as a Matérn kernel) is estimated by maximum likelihood. We show that the maximum likelihood estimation of the scale parameter alone provides significant adaptat…
We propose a new testing procedure about the tail weight parameter of multivariate Student $t$ distributions by having recourse to the Le Cam methodology. Our test is asymptotically as efficient as the classical likelihood ratio test, but outperforms the latter by its flexibility and simplicity: indeed, our approach allows to estimate the location and scatter nuisance parameters by any root-$n$ consistent estimators, hereby avoiding numerically complex maximum likelihood estimation. The finite-sample properties of our test are analyzed in a Monte Carlo simulation study, and we apply our method…
This survey provides a self-contained account of $M$-estimation of multivariate scatter. In particular, we present new proofs for existence of the underlying $M$-functionals and discuss their weak continuity and differentiability. This is done in a rather general framework with matrix-valued random variables. By doing so we reveal a connection between Tyler's (1987) $M$-functional of scatter and the estimation of proportional covariance matrices. Moreover, this general framework allows us to treat a new class of scatter estimators, based on symmetrizations of arbitrary order. Finally these res…
The method of maximum likelihood estimation (MLE) is a widely used statistical approach for estimating the values of one or more unknown parameters of a probabilistic model based on observed data. In this tutorial, I briefly review the mathematical foundations of MLE, then reformulate the problem for the measurement of a spatially-varying optical intensity distribution. In this context, the detection of each individual photon is treated as a random event, the outcome being the photon's location. A typical measurement consists of many detected photons, which accumulate to form a spatial intensi…
Assumptions on a likelihood function, including a local Glivenko-Cantelli condition, imply the existence of M-estimators converging to an M-functional. Scatter matrix-valued estimators, defined on all empirical measures on ${\Bbb{R}}^d$ for $d\geq 2$, and equivariant under all, including singular, affine transformations, are shown to be constants times the sample covariance matrix. So, if weakly continuous, they must be identically 0. Results are stated on existence and differentiability of location and scatter functionals, defined on a weakly dense, weakly open set of laws, via elliptically s…
Due to its heavy-tailed and fully parametric form, the multivariate generalized Gaussian distribution (MGGD) has been receiving much attention for modeling extreme events in signal and image processing applications. Considering the estimation issue of the MGGD parameters, the main contribution of this paper is to prove that the maximum likelihood estimator (MLE) of the scatter matrix exists and is unique up to a scalar factor, for a given shape parameter β\in(0,1). Moreover, an estimation algorithm based on a Newton-Raphson recursion is proposed for computing the MLE of MGGD parameters. Variou…
In this paper, we consider maximum likelihood estimations of the degree of freedom parameter $ν$, the location parameter $μ$ and the scatter matrix $Σ$ of the multivariate Student-$t$ distribution. In particular, we are interested in estimating the degree of freedom parameter $ν$ that determines the tails of the corresponding probability density function and was rarely considered in detail in the literature so far. We prove that under certain assumptions a minimizer of the negative log-likelihood function exists, where we have to take special care of the case $ν\rightarrow \infty$, for which t…
