We deal with the equivariant estimation of scatter and location for p-dimensional data, giving emphasis to scatter. It it important that the estimators possess both a high efficiency for normal data and a high resistance to outliers, that is, a low bias under contamination. The most frequently employed estimators are not quite satisfactory in this respect. The Minimum Volume Ellipsoid (MVE) and Minimum Covariance Determinant (MCD) estimators are known to have a very low efficiency. S-Estimators (Davies 1987) with a monotonic weight function like the bisquare behave satisfactorily for "small" p, say p not larger than 10. Rocke (1996) showed that their efficiency tends to one with increasing p. Unfortunately, this advantage is paid with a serious loss of robustness for large p. We consider three families of estimators with controllable efficiencies: non-monotonic S-estimators (Rocke 1996), MM-estimators (Tatsuoka and Tyler 2000) and tau-estimators (Lopuhaa 1991), whose performance for large p has not been explored to date. Two types of starting estimators are employed: the MVE computed through subsampling, and a semi-deterministic procedure proposed by Peña and Prieto (2007) for outlier detection. A simulation study shows that the Rocke and MM estimators starting from the Peña-Prieto estimator and with an adequate tuning, can simultaneously attain high efficiency and high robustness.
Multivariate location and scatter matrix estimation is a cornerstone in multivariate data analysis. We consider this problem when the data may contain independent cellwise and casewise outliers. Flat data sets with a large number of variables and a relatively small number of cases are common place in modern statistical applications. In these cases global down-weighting of an entire case, as performed by traditional robust procedures, may lead to poor results. We highlight the need for a new generation of robust estimators that can efficiently deal with cellwise outliers and at the same time sh…
We introduce a robust estimator of the location parameter for the change-point in the mean based on the Wilcoxon statistic and establish its consistency for $L_1$ near epoch dependent processes. It is shown that the consistency rate depends on the magnitude of change. A simulation study is performed to evaluate finite sample properties of the Wilcoxon-type estimator in standard cases, as well as under heavy-tailed distributions and disturbances by outliers, and to compare it with a CUSUM-type estimator. It shows that the Wilcoxon-type estimator is equivalent to the CUSUM-type estimator in stan…
Schoenberg transformations, mapping Euclidean configurations into Euclidean configurations, define in turn a transformed inertia, whose minimization produces robust location estimates. The procedure only depends upon Euclidean distances between observations, and applies equivalently to univariate and multivariate data. The choice of the family of transformations and their parameters defines a flexible location strategy, generalizing M-estimators. Two regimes of solutions are identified. Theoretical results on their existence and stability are provided, and illustrated on two data sets.
Covariance matrix estimation is one of the most important problems in statistics. To accommodate the complexity of modern datasets, it is desired to have estimation procedures that not only can incorporate the structural assumptions of covariance matrices, but are also robust to outliers from arbitrary sources. In this paper, we define a new concept called matrix depth and then propose a robust covariance matrix estimator by maximizing the empirical depth function. The proposed estimator is shown to achieve minimax optimal rate under Huber's $ε$-contamination model for estimating covariance/sc…
University of Chicago（芝加哥大学）；Hong Kong University of Science and Technology（香港科技大学）
Robust scatter estimation is a fundamental task in statistics. The recent discovery on the connection between robust estimation and generative adversarial nets (GANs) by Gao et al. (2018) suggests that it is possible to compute depth-like robust estimators using similar techniques that optimize GANs. In this paper, we introduce a general learning via classification framework based on the notion of proper scoring rules. This framework allows us to understand both matrix depth function and various GANs through the lens of variational approximations of $f$-divergences induced by proper scoring ru…
Robust estimation of location and concentration parameters for the von Mises-Fisher distribution is discussed. A key reparametrisation is achieved by expressing the two parameters as one vector on the Euclidean space. With this representation, we first show that maximum likelihood estimator for the von Mises-Fisher distribution is not robust in some situations. Then we propose two families of robust estimators which can be derived as minimisers of two density power divergences. The presented families enable us to estimate both location and concentration parameters simultaneously. Some properti…
In the multivariate one-sample location model, we propose a class of flexible robust, affine-equivariant L-estimators of location, for distributions invoking affine-invariance of Mahalanobis distances of individual observations. An involved iteration process for their computation is numerically illustrated.
We consider component-wise estimation of order restricted location/scale parameters of a general bivariate location/scale distribution under the generalized Pitman nearness criterion (GPN). We develop some general results that, in many situations, are useful in finding improvements over location/scale equivariant estimators. In particular, under certain conditions, these general results provide improvements over the unrestricted Pitman nearest location/scale equivariant estimators and restricted maximum likelihood estimators. The usefulness of the obtained results is illustrated through their …
We consider component-wise equivariant estimation of order restricted location/scale parameters of a general bivariate distribution under quite general conditions on underlying distributions and the loss function. This paper unifies various results in the literature dealing with sufficient conditions for finding improvments over arbitrary location/scale equivariant estimators. The usefulness of these results is illustrated through various examples. A simulation study is considered to compare risk performances of various estimators under bivariate normal and independent gamma probability models…
In a missing-data setting, we have a sample in which a vector of explanatory variables x_i is observed for every subject i, while scalar outcomes y_i are missing by happenstance on some individuals. In this work we propose robust estimates of the distribution of the responses assuming missing at random (MAR) data, under a semiparametric regression model. Our approach allows the consistent estimation of any weakly continuous functional of the response's distribution. In particular, strongly consistent estimates of any continuous location functional, such as the median or MM functionals, are pro…
We consider the problem of parameter estimation in the case of observation of the trajectory of diffusion process. We suppose that the drift coefficient has a singularity of cusp-type and the unknown parameter corresponds to the position of the point of the cusp. The asymptotic properties of the maximum likelihood estimator and Bayesian estimators are described in the asymptotics of {\it small noise}, i.e., as the diffusion coefficient tends to zero. The consistency, limit distributions and the convergence of moments of these estimators are established.
This article studies the weak convergence and associated Central Limit Theorem for blurring and nonblurring processes. Then, they are applied to the estimation of location parameter. Simulation studies show that the location estimation based on the convergence point of blurring process is more robust and often more efficient than that of nonblurring process.
Density estimation represents one of the most successful applications of Bayesian nonparametrics. In particular, Dirichlet process mixtures of normals are the gold standard for density estimation and their asymptotic properties have been studied extensively, especially in the univariate case. However a gap between practitioners and the current theoretical literature is present. So far, posterior asymptotic results in the multivariate case are available only for location mixtures of Gaussian kernels with independent prior on the common covariance matrix, while in practice as well as from a conc…
We investigate predictive density estimation under the $L^2$ Wasserstein loss for location families and location-scale families. We show that plug-in densities form a complete class and that the Bayesian predictive density is given by the plug-in density with the posterior mean of the location and scale parameters. We provide Bayesian predictive densities that dominate the best equivariant one in normal models.
We consider an inhomogeneous Poisson process $X$ on $[0,T]$. The intensity function of $X$ is supposed to be strictly positive and smooth on $[0,T]$ except at the point $θ$, in which it has either a 0-type singularity (tends to 0 like $\abs{x}^p$, $p\in(0,1)$), or an $\infty$-type singularity (tends to $\infty$ like $\abs{x}^p$, $p\in(-1,0)$). We suppose that we know the shape of the intensity function, but not the location of the singularity. We consider the problem of estimation of this location (shift) parameter $θ$ based on $n$ observations of the process $X$. We study the Bayesian estimat…
We present a review of some recent results on estimation of location parameter for several models of observations with cusp-type singularity at the change point. We suppose that the cusp-type models fit better to the real phenomena described usually by change point models. The list of models includes Gaussian, inhomogeneous Poisson, ergodic diffusion processes, time series and the classical case of i.i.d. observations. We describe the properties of the maximum likelihood and Bayes estimators under some asymptotic assumptions. The asymptotic efficiency of estimators are discussed as well and th…
This paper investigates estimation of the mean vector under invariant quadratic loss for a spherically symmetric location family with a residual vector with density of the form $ f(x,u)=η^{(p+n)/2}f(η\{\|x-θ\|^2+\|u\|^2\}) $, where $η$ is unknown. We show that the natural estimator $x$ is admissible for $p=1,2$. Also, for $p\geq 3$, we find classes of generalized Bayes estimators that are admissible within the class of equivariant estimators of the form $\{1-ξ(x/\|u\|)\}x$. In the Gaussian case, a variant of the James--Stein estimator, $[1-\{(p-2)/(n+2)\}/\{\|x\|^2/\|u\|^2+(p-2)/(n+2)+1\}]x$, …
Assumptions on a likelihood function, including a local Glivenko-Cantelli condition, imply the existence of M-estimators converging to an M-functional. Scatter matrix-valued estimators, defined on all empirical measures on ${\Bbb{R}}^d$ for $d\geq 2$, and equivariant under all, including singular, affine transformations, are shown to be constants times the sample covariance matrix. So, if weakly continuous, they must be identically 0. Results are stated on existence and differentiability of location and scatter functionals, defined on a weakly dense, weakly open set of laws, via elliptically s…
The problem of simultaneous estimation of location/scale parameters $θ_1$ and $θ_2$ of a general bivariate location/scale model, when the ordering between the parameters is known apriori (say, $θ_1\leq θ_2$), has been considered. We consider isotonic regression estimators based on the best location/scale equivariant estimators (BLEEs/BSEEs) of $θ_1$ and $θ_2$ with general weight functions. Let $\mathcal{D}$ denote the corresponding class of isotonic regression estimators of $(θ_1,θ_2)$. Under the sum of the weighted squared error loss function, we characterize admissible estimators within the …
The problem of estimating location (scale) parameters $θ_1$ and $θ_2$ of two distributions when the ordering between them is known apriori (say, $θ_1\leq θ_2$) has been extensively studied in the literature. Many of these studies are centered around deriving estimators that dominate the best location (scale) equivariant estimators, for the unrestricted case, by exploiting the prior information that $θ_1 \leq θ_2$. Several of these studies consider specific distributions such that the associated random variables are statistically independent. This paper considers a general bivariate model and g…
The paper aims at finding widely and smoothly defined nonparametric location and scatter functionals. As a convenient vehicle, maximum likelihood estimation of the location vector m and scatter matrix S of an elliptically symmetric t distribution on d-dimensional space with degrees of freedom larger than 1 extends to an M-functional defined on all probability distributions P in a weakly open, weakly dense domain U. Here U consists of P not putting too much mass in hyperplanes of dimension < d, as shown for empirical measures by Kent and Tyler, Ann. Statist. 1991. It is shown here that (m,S) is…
Consider an i.i.d. sample X^*_1,X^*_2,...,X^*_n from a location-scale family, and assume that the only available observations consist of the partial maxima (or minima)sequence, X^*_{1:1},X^*_{2:2},...,X^*_{n:n}, where X^*_{j:j}=max{X^*_1,...,X^*_j}. This kind of truncation appears in several circumstances, including best performances in athletics events. In the case of partial maxima, the form of the BLUEs (best linear unbiased estimators) is quite similar to the form of the well-known Lloyd's (1952, Least-squares estimation of location and scale parameters using order statistics, Biometrika, …
We consider the problem of multivariate location and scatter matrix estimation when the data contain cellwise and casewise outliers. Agostinelli et al. (2015) propose a two-step approach to deal with this problem: first, apply a univariate filter to remove cellwise outliers and second, apply a generalized S-estimator to downweight casewise outliers. We improve this proposal in three main directions. First, we introduce a consistent bivariate filter to be used in combination with the univariate filter in the first step. Second, we propose a new fast subsampling procedure to generate starting po…
Let $θ$ and $μ$ denote the location and the size of the mode of a probability density. We study the joint convergence rates of semirecursive kernel estimators of $θ$ and $μ$. We show how the estimation of the size of the mode allows to measure the relevance of the estimation of its location. We also enlighten that, beyond their computational advantage on nonrecursive estimators, the semirecursive estimators are preferable to use for the construction on confidence regions.
We consider the problem of robust mean and location estimation w.r.t. any pseudo-norm of the form $x\in\mathbb{R}^d\to ||x||_S = \sup_{v\in S}<v,x>$ where $S$ is any symmetric subset of $\mathbb{R}^d$. We show that the deviation-optimal minimax subgaussian rate for confidence $1-δ$ is $$ \max\left(\frac{l^*(Σ^{1/2}S)}{\sqrt{N}}, \sup_{v\in S}||Σ^{1/2}v||_2\sqrt{\frac{\log(1/δ)}{N}}\right) $$where $l^*(Σ^{1/2}S)$ is the Gaussian mean width of $Σ^{1/2}S$ and $Σ$ the covariance of the data (in the benchmark i.i.d. Gaussian case). This improves the entropic minimax lower bound from [Lugosi and Men…
Modeling and understanding multivariate extreme events is challenging, but of great importance in various applications - e.g. in biostatistics, climatology, and finance. The separating Hill estimator can be used in estimating the extreme value index of a heavy tailed multivariate elliptical distribution. We consider the asymptotic behavior of the separating Hill estimator under estimated location and scatter. The asymptotic properties of the separating Hill estimator are known under elliptical distribution with known location and scatter. However, the effect of estimation of the location and s…
Let $\{P_θ:θ\in {\mathbb R}^d\}$ be a log-concave location family with $P_θ(dx)=e^{-V(x-θ)}dx,$ where $V:{\mathbb R}^d\mapsto {\mathbb R}$ is a known convex function and let $X_1,\dots, X_n$ be i.i.d. r.v. sampled from distribution $P_θ$ with an unknown location parameter $θ.$ The goal is to estimate the value $f(θ)$ of a smooth functional $f:{\mathbb R}^d\mapsto {\mathbb R}$ based on observations $X_1,\dots, X_n.$ In the case when $V$ is sufficiently smooth and $f$ is a functional from a ball in a Hölder space $C^s,$ we develop estimators of $f(θ)$ with minimax optimal error rates measured by…
In reliability and life testing studies, the topic of estimating hazard rate has received great attention in recent years since an estimate of hazard rate is a quite useful tool for making decisions. Some works have included nonparametric approaches while some have considered parametric structural models for complete as well as censored data sets; see Meeker et al. (1992), Antoniadis and Grégoire (1999), Rai and Singh (2003), Bezandry et al. (2005), Brunel and Comte (2008), and Mahapatra et al. (2012). Depending on the shapes of the hazard rate, efficiencies differ markedly across proposed est…
Consider two independent exponential populations having different unknown location parameters and a common unknown scale parameter. Call the population associated with the larger location parameter as the "best" population and the population associated with the smaller location parameter as the "worst" population. For the goal of selecting the best (worst) population a natural selection rule, that has many optimum properties, is the one which selects the population corresponding to the larger (smaller) minimal sufficient statistic. In this article, we consider the problem of estimating the loc…
For normal canonical models, and more generally a vast array of general spherically symmetric location-scale models with a residual vector, we consider estimating the (univariate) location parameter when it is lower bounded. We provide conditions for estimators to dominate the benchmark minimax MRE estimator, and thus be minimax under scale invariant loss. These minimax estimators include the generalized Bayes estimator with respect to the truncation of the common non-informative prior onto the restricted parameter space for normal models under general convex symmetric loss, as well as non-nor…
We revisit the problem of estimating the center of symmetry $θ$ of an unknown symmetric density $f$. Although Stone (1975), Van Eden (1970), and Sacks (1975) constructed adaptive estimators of $θ$ in this model, their estimators depend on tuning parameters. In an effort to circumvent the dependence on tuning parameters, we impose an additional assumption of log-concavity on $f$. We show that in this shape-restricted model, the maximum likelihood estimator (MLE) of $θ$ exists. We also study some truncated one-step estimators and show that they are $\sqrt{n}-$consistent, and nearly achieve the a…
Robust estimators of location and dispersion are often used in the elliptical model to obtain an uncontaminated and highly representative subsample by trimming the data outside an ellipsoid based in the associated Mahalanobis distance. Here we analyze some one (or $k$)-step Maximum Likelihood Estimators computed on a subsample obtained with such a procedure. We introduce different models which arise naturally from the ways in which the discarded data can be treated, leading to truncated or censored likelihoods, as well as to a likelihood based on an only outliers gross errors model. Results on…
Marked point processes intensity estimation using sparse group Lasso method applied to locations of lucrative and cooperative banks in mainland France
Univ. Grenoble Alpes（格勒诺布尔阿尔卑斯大学）；CNRS（法国国家科学研究中心）；LJK（拉格朗日数学与计算实验室）；Sciences Po Grenoble, PACTE（格勒诺布尔政治学院PACTE研究所）；Department of Statistics, Institut Teknologi Sepuluh Nopember（十·一理工学院统计学系）
The University of Texas at Austin（德克萨斯大学奥斯汀分校）；University of Wisconsin–Madison（威斯康星大学麦迪逊分校）；Purdue University（普渡大学）
We consider 1-dimensional location estimation, where we estimate a parameter $λ$ from $n$ samples $λ+ η_i$, with each $η_i$ drawn i.i.d. from a known distribution $f$. For fixed $f$ the maximum-likelihood estimate (MLE) is well-known to be optimal in the limit as $n \to \infty$: it is asymptotically normal with variance matching the Cramér-Rao lower bound of $\frac{1}{n\mathcal{I}}$, where $\mathcal{I}$ is the Fisher information of $f$. However, this bound does not hold for finite $n$, or when $f$ varies with $n$. We show for arbitrary $f$ and $n$ that one can recover a similar theory based on…
This note considers the problem of approximating the locations of dominant spikes for a probability measure from noisy spectrum measurements under the condition of residue signal, significant noise level, and no minimum spectrum separation. We show that the simple procedure of thresholding the smoothed inverse Fourier transform allows for approximating the spike locations rather accurately.
This paper studies the optimal rate of estimation in a finite Gaussian location mixture model in high dimensions without separation conditions. We assume that the number of components $k$ is bounded and that the centers lie in a ball of bounded radius, while allowing the dimension $d$ to be as large as the sample size $n$. Extending the one-dimensional result of Heinrich and Kahn \cite{HK2015}, we show that the minimax rate of estimating the mixing distribution in Wasserstein distance is $Θ((d/n)^{1/4} + n^{-1/(4k-2)})$, achieved by an estimator computable in time $O(nd^2+n^{5/4})$. Furthermor…
The University of Texas at Austin（德克萨斯大学奥斯汀分校）；University of Wisconsin-Madison（威斯康星大学麦迪逊分校）
In location estimation, we are given $n$ samples from a known distribution $f$ shifted by an unknown translation $λ$, and want to estimate $λ$ as precisely as possible. Asymptotically, the maximum likelihood estimate achieves the Cramér-Rao bound of error $\mathcal N(0, \frac{1}{n\mathcal I})$, where $\mathcal I$ is the Fisher information of $f$. However, the $n$ required for convergence depends on $f$, and may be arbitrarily large. We build on the theory using \emph{smoothed} estimators to bound the error for finite $n$ in terms of $\mathcal I_r$, the Fisher information of the $r$-smoothed di…
In this paper, we consider maximum likelihood estimations of the degree of freedom parameter $ν$, the location parameter $μ$ and the scatter matrix $Σ$ of the multivariate Student-$t$ distribution. In particular, we are interested in estimating the degree of freedom parameter $ν$ that determines the tails of the corresponding probability density function and was rarely considered in detail in the literature so far. We prove that under certain assumptions a minimizer of the negative log-likelihood function exists, where we have to take special care of the case $ν\rightarrow \infty$, for which t…
