Both parametric distribution functions appearing in extreme value theory - the generalized extreme value distribution and the generalized Pareto distribution - have log-concave densities if the extreme value index gamma is in [-1,0]. Replacing the order statistics in tail index estimators by their corresponding quantiles from the distribution function that is based on the estimated log-concave density leads to novel smooth quantile and tail index estimators. These new estimators aim at estimating the tail index especially in small samples. Acting as a smoother of the empirical distribution function, the log-concave distribution function estimator reduces estimation variability to a much greater extent than it introduces bias. As a consequence, Monte Carlo simulations demonstrate that the smoothed version of the estimators are well superior to their non-smoothed counterparts, in terms of mean squared error.
Estimating the tail index parameter is one of the primal objectives in extreme value theory. For heavy-tailed distributions the Hill estimator is the most popular way to estimate the tail index parameter. Improving the Hill estimator was aimed by recent works with different methods, for example by using bootstrap, or Kolmogorov-Smirnov metric. These methods are asymptotically consistent, but for tail index $ξ>1$ and smaller sample sizes the estimation fails to approach the theoretical value for realistic sample sizes. In this paper, we introduce a new empirical method, which can estimate high …
We present a nonparametric family of estimators for the tail index of a Pareto-type distribution when covariate information is available. Our estimators are based on a weighted sum of the log-spacings between some selected observations. This selection is achieved through a moving window approach on the covariate domain and a random threshold on the variable of interest. Asymptotic normality is proved under mild regularity conditions and illustrated for some weight functions. Finite sample performances are presented on a real data study.
We show that the full-sample bootstrap is asymptotically valid for constructing confidence intervals for high-quantiles, tail probabilities, and other tail parameters of a univariate distribution. This resolves the doubts that have been raised about the validity of such bootstrap methods. In our extensive simulation study, the overall performance of the bootstrap method was better than that of the standard asymptotic method, indicating that the bootstrap method is at least as good, if not better than, the asymptotic method for inference. This paper also lays the foundation for developing boots…
We propose a class of weighted least squares estimators for the tail index of a distribution function with a regularly varying upper tail. Our approach is based on the method developed by \cite{Holan2010} for the Parzen tail index. Asymptotic normality of the estimators is proved. Through a simulation study, these and earlier estimators are compared in the Pareto and Hall models using the mean squared error as criterion. The results show that the weighted least squares estimator is better than the other estimators investigated.
This article is devoted to the study of tail index estimation based on i.i.d. multivariate observations, drawn from a standard heavy-tailed distribution, i.e. of which 1-d Pareto-like marginals share the same tail index. A multivariate Central Limit Theorem for a random vector, whose components correspond to (possibly dependent) Hill estimators of the common shape index alpha, is established under mild conditions. Motivated by the statistical analysis of extremal spatial data in particular, we introduce the concept of (standard) heavy-tailed random field of tail index alpha and show how this l…
We consider heavy-tailed distributions and compare the well-known estimators of the tail index, based on extreme value theory with a comparatively recent estimator based on a different idea.
Estimation of the tail index of heavy-tailed distributions and its applications are essential in many research areas. We propose a class of weighted least squares (WLS) estimators for the Parzen tail index. Our approach is based on the method developed by \cite{Holan2010}. We investigate consistency and asymptotic normality of the WLS estimators. Through a simulation study, we make a comparison with the Hill, Pickands, DEdH (Dekkers, Einmahl and de Haan) and ordinary least squares (OLS) estimators using the mean square error as criterion. The results show that in a restricted model some member…
A tail empirical process for heavy-tailed and right-censored data is introduced and its Gaussian approximation is established. In this context, a (weighted) new Hill-type estimator for positive extreme value index is proposed and its consistency and asymptotic normality are proved by means of the aforementioned process in the framework of second-order conditions of regular variation. In a comparative simulation study, the newly defined estimator is seen to perform better than the already existing ones in terms of both bias and mean squared error. As a real data example, we apply our estimation…
On the basis of Nelson-Aalen nonparametric estimator of the cumulative distribution function, we provide a weak approximation to tail product-limit process for randomly right-censored heavy-tailed data. In this context, a new consistent reduced-bias estimator of the extreme value index is introduced and its asymptotic normality is established only by assuming the second-order regular variation of the underlying distribution function. A simulation study shows that the newly proposed estimator performs better than the existing ones.
This paper investigates pooling strategies for tail index and extreme quantile estimation from heavy-tailed data. To fully exploit the information contained in several samples, we present general weighted pooled Hill estimators of the tail index and weighted pooled Weissman estimators of extreme quantiles calculated through a nonstandard geometric averaging scheme. We develop their large-sample asymptotic theory across a fixed number of samples, covering the general framework of heterogeneous sample sizes with different and asymptotically dependent distributions. Our results include optimal ch…
It was shown that when one disposes of a parametric information of the truncation distribution, the semiparametric estimator of the distribution function for truncated data (Wang, 1989) is more efficient than the nonparametric one. On the basis of this estimation method, we derive an estimator for the tail index of Pareto-type distributions that are randomly right-truncated and establish its consistency and asymptotic normality. The finite sample behavior of the proposed estimator is carried out by simulation study. We point out that, in terms of both bias and root of the mean squared error, o…
In this paper, we define a kernel estimator for the tail index of a Pareto-type distribution under random right-truncation and establish its asymptotic normality. A simulation study shows that, compared to the estimators recently proposed by Gardes & Stupfler (2015) and Benchaira et al. (2015), this newly introduced estimator behaves better, in terms of bias and mean squared error, for small samples.
In the paper we propose some new class of functions which is used to construct tail index estimators. Functions from this new class is non-monotone in general, but presents a product of two monotone functions: the power function and the logarithmic function, which plays essential role in the classical Hill estimator. Introduced new estimators have better asymptotic performance comparing with the Hill estimator and other popular estimators over all range of the parameters present in the second order regular variation condition. Asymptotic normality of the introduced estimators is proved, and co…
In this paper we continue the investigation of the SRCEN estimator of the extreme value index $γ$ (or the tail index $α=1/γ$) proposed in \cite{MCE} for $γ>1/2$. We propose a new estimator based on the local maximum. This, in fact, is a modification of the SRCEN estimator to the case $γ>0$. We establish the consistency and asymptotic normality of the newly proposed estimator for i.i.d. data. Also, a short discussion on the comparison of the estimators is included.
We are trying to give a mathematically correct definition of outliers. Our approach is based on the distance between two last order statistics and appears to be connected to the law of large numbers. Key words: outliers, law of large numbers, heavy tails, stability index.
This paper presents two results concerning uniform confidence intervals for the tail index and the extreme quantile. First, we show that it is impossible to construct a length-optimal confidence interval satisfying the correct uniform coverage over a local non-parametric family of tail distributions. Second, in light of the impossibility result, we construct honest confidence intervals that are uniformly valid by incorporating the worst-case bias in the local non-parametric family. The proposed method is applied to simulated data and a real data set of National Vital Statistics from National C…
In this paper, we propose an estimator of the second-order parameter of randomly right-truncated Pareto-type distributions data and establish its consistency and asymptotic normality. Moreover, we derive an asymptotically unbiased estimator of the tail index and study its asymptotic behaviour. Our considerations are based on a useful Gaussian approximation of the tail product-limit process recently given by Benchaira et al. [Tail product-limit process for truncated data with application to extreme value index estimation. Extremes, 2016; 19: 219-251] and the results of Gomes et al. [Semi-parame…
We make use of the empirical process theory to approximate the adapted Hill estimator, for censored data, in terms of Gaussian processes. Then, we derive its asymptotic normality, only under the usual second-order condition of regular variation. Our methodology allows to relax the assumptions, made in Einmahl, Fils-Villetard and Guillou(2008), on the heavy-tailed distribution functions and the sample fraction of upper order statistics.
We study a new estimator for the tail index of a distribution in the Frechet domain of attraction that arises naturally by computing subsample maxima. This estimator is equivalent to taking a U-statistic over a Hill estimator with two order statistics. The estimator presents multiple advantages over the Hill estimator. In particular, it has asymptotically smooth sample paths as a function of the threshold k, making it considerably more stable than the Hill estimator. The estimator also admits a simple and intuitive threshold selection rule that does not require fitting a second-order model. Jo…
By means of a Lynden-Bell integral with deterministic threshold, Worms and Worms [A Lynden-Bell integral estimator for extremes of randomly truncated data. Statist. Probab. Lett. 2016; 109: 106-117] recently introduced an asymptotically normal estimator of the tail index for randomly right-truncated Pareto-type data. In this context, we consider the random threshold case to derive a Hill-type estimator and establish its consistency and asymptotic normality. A simulation study is carried out to evaluate the finite sample behavior of the proposed estimator.
This work develops the asymptotic properties (weak consistency and Gaussianity), in the high-frequency limit, of approximate maximum likelihood estimators for the spectral parameters of Gaussian and isotropic spherical random fields. The procedure we used exploits the so-called mexican needlet construction by Geller and Mayeli in [Geller, Mayeli (2009)]. Furthermore, we propose a plug-in procedure to optimize the precision of the estimators in terms of asymptotic variance.
Financial crises are a recurrent phenomenon with important effects on the real economy. The financial system is inherently fragile and it is therefore of great importance to be able to measure and characterize its systemic stability. Multivariate extreme value theory provide us such a framework through the \emph{fragility index} (Geluk \cite{gel+}, \emph{et al.}, 2007; Falk and Tichy, \cite{falk+tichy1,falk+tichy2} 2010, 2011). Here we generalize this concept and contribute to the modeling of the stability of a stochastic system divided into blocks. We will find several relations with well-kno…
A novel statistical method is proposed and investigated for estimating a heavy tailed density under mild smoothness assumptions. Statistical analyses of heavy-tailed distributions are susceptible to the problem of sparse information in the tail of the distribution getting washed away by unrelated features of a hefty bulk. The proposed Bayesian method avoids this problem by incorporating smoothness and tail regularization through a carefully specified semiparametric prior distribution, and is able to consistently estimate both the density function and its tail index at near minimax optimal rate…
We introduce a kernel estimator, to the tail index of a right-censored Pareto-type distribution, that generalizes Worms's one (Worms and Worms, 2014)in terms of weight coefficients. Under some regularity conditions, the asymptotic normality of the proposed estimator is established. In the framework of the second-order condition, we derive an asymptotically bias-reduced version to the new estimator. Through a simulation study, we conclude that one of the main features of the proposed kernel estimator is its smoothness contrary to Worms's one, which behaves, rather erratically, as a function of …
This paper presents an adaptive version of the Hill estimator based on Lespki's model selection method. This simple data-driven index selection method is shown to satisfy an oracle inequality and is checked to achieve the lower bound recently derived by Carpentier and Kim. In order to establish the oracle inequality, we derive non-asymptotic variance bounds and concentration inequalities for Hill estimators. These concentration inequalities are derived from Talagrand's concentration inequality for smooth functions of independent exponentially distributed random variables combined with three to…
Empirical likelihood is a well-known nonparametric method in statistics and has been widely applied in statistical inference. The method has been employed by Lu and Peng (2002) to constructing confidence intervals for the tail index of a heavy-tailed distribution. It is demonstrated in Lu and Peng (2002) that the empirical likelihood-based confidence intervals performs better than confidence intervals based on normal approximation in terms of the coverage probability. In general, the empirical likelihood method can be hindered by its imprecision in the coverage probability when the sample size…
Graphex processes resolve some pathologies in traditional random graph models, notably, providing models that are both projective and allow sparsity. Most of the literature on graphex processes study them from a probabilistic point of view. Techniques for inferring the parameter of these processes -- the so-called \textit{graphon} -- are still marginal; exceptions are a few papers considering parametric families of graphons. Nonparametric estimation remains unconsidered. In this paper, we propose estimators for a selected choice of functionals of the graphon. Our estimators originate from the …
In this paper, we introduce reduced-bias estimators for the estimation of the tail index of a Pareto-type distribution. This is achieved through the use of a regularised weighted least squares with an exponential regression model for log-spacings of top order statistics. The asymptotic properties of the proposed estimators are investigated analytically and found to be asymptotically unbiased, consistent and normally distributed. Also, the finite sample behaviour of the estimators are studied through a simulations theory. The proposed estimators were found to yield low bias and MSE. In addition…
Modern statistical analyses often encounter datasets with massive sizes and heavy-tailed distributions. For datasets with massive sizes, traditional estimation methods can hardly be used to estimate the extreme value index directly. To address the issue, we propose here a subsampling-based method. Specifically, multiple subsamples are drawn from the whole dataset by using the technique of simple random subsampling with replacement. Based on each subsample, an approximate maximum likelihood estimator can be computed. The resulting estimators are then averaged to form a more accurate one. Under …
In this paper, we propose a reduced-bias estimator of the EVI for Pareto-type tails (heavy-tailed) distributions. This is derived using the weighted least squares method. It is shown that the estimator is unbiased, consistent and asymptotically normal under the second-order conditions on the underlying distribution of the data. The finite sample properties of the proposed estimator are studied through a simulation study. The results show that it is competitive to the existing estimators of the extreme value index in terms of bias and Mean Square Error. In addition, it yields estimates of $γ>0$…
Estimating the tail index of Pareto-type distributions from geometric records
Universidad de Zaragoza（萨拉戈萨大学）；Instituto de Biocomputación y Física de Sistemas Complejos (BIFI), Universidad de Zaragoza（萨拉戈萨大学生物计算与复杂系统物理研究所 (BIFI)）；Universidad de Chile（智利大学）
Heavy tailed phenomena are naturally analyzed by extreme value statistics. A crucial step in such an analysis is the estimation of the extreme value index, which describes the tail heaviness of the underlying probability distribution. We consider the situation where we have next to the $n$ observations of interest another $n+m$ observations of one or more related variables, like, e.g., financial losses due to earthquakes and the related amounts of energy released, for a longer period than that of the losses. Based on such a data set, we present an adapted version of the Hill estimator that sho…
In several different fields, there is interest in analyzing the upper or lower tail quantile of the underlying distribution rather than mean or center quantile. However, the investigation of the tail quantile is difficult because of data sparsity. In this paper, we attempt to develop nonparametric quantile regression for the extremal quantile level. In extremal quantile regression, there are two types of technical conditions of the order of convergence of the quantile level: intermediate order or extreme order. For the intermediate order quantile, the ordinary nonparametric estimator is used. …
This work deals with the estimation of the extreme value index and extreme quantiles for heavy tailed data,randomly right truncated by another heavy tailed variable. Under mild assumptions and the condition thatthe truncated variable is less heavy-tailed than the truncating variable, asymptotic normality is proved for bothestimators. The proposed estimator of the extreme value index is an adaptation of the Hill estimator, in thenatural form of a Lynden-Bell integral. Simulations illustrate the quality of the estimators under a variety ofsituations.
The paper suggests a simple method of deriving minimax lower bounds to the accuracy of statistical inference on heavy tails. A well-known result by Hall and Welsh (Ann. Statist. 12 (1984) 1079-1084) states that if $\hatα_n$ is an estimator of the tail index $α_P$ and $\{z_n\}$ is a sequence of positive numbers such that $\sup_{P\in{\mathcal{D}}_r}\mathbb{P}(|\hatα_n-α_P|\ge z_n)\to0$, where ${\mathcal{D}}_r$ is a certain class of heavy-tailed distributions, then $z_n\gg n^{-r}$. The paper presents a non-asymptotic lower bound to the probabilities $\mathbb{P}(|\hatα_n-α_P|\ge z_n)$. We also est…
We consider estimation and inference in a single index regression model with an unknown but smooth link function. In contrast to the standard approach of using kernels or regression splines, we use smoothing splines to estimate the smooth link function. We develop a method to compute the penalized least squares estimators (PLSEs) of the parametric and the nonparametric components given independent and identically distributed (i.i.d.)~data. We prove the consistency and find the rates of convergence of the estimators. We establish asymptotic normality under under mild assumption and prove asympt…
Estimating the structures at high or low quantiles has become an important subject and attracted increasing attention across numerous fields. However, due to data sparsity at tails, it usually is a challenging task to obtain reliable estimation, especially for high-dimensional data. This paper suggests a flexible parametric structure to tails, and this enables us to conduct the estimation at quantile levels with rich observations and then to extrapolate the fitted structures to far tails. The proposed model depends on some quantile indices and hence is called the quantile index regression. Mor…
We study the problems related to the estimation of the Gini index in presence of a fat-tailed data generating process, i.e. one in the stable distribution class with finite mean but infinite variance (i.e. with tail index $α\in(1,2)$). We show that, in such a case, the Gini coefficient cannot be reliably estimated using conventional nonparametric methods, because of a downward bias that emerges under fat tails. This has important implications for the ongoing discussion about economic inequality. We start by discussing how the nonparametric estimator of the Gini index undergoes a phase transiti…
The so-called partition function is a sample moment statistic based on blocks of data and it is often used in the context of multifractal processes. It will be shown that its behaviour is strongly influenced by the tail of the distribution underlying the data either in i.i.d. and weakly dependent cases. These results will be exploited to develop graphical and estimation methods for the tail index of a distribution. The performance of the tools proposed is analyzed and compared with other methods by means of simulations and examples.
Asymptotic theory of tail index estimation has been studied extensively in the frequentist literature on extreme values, but rarely in the Bayesian context. We investigate whether popular Bayesian kernel mixture models are able to support heavy tailed distributions and consistently estimate the tail index. We show that posterior inconsistency in tail index is surprisingly common for both parametric and nonparametric mixture models. We then present a set of sufficient conditions under which posterior consistency in tail index can be achieved, and verify these conditions for Pareto mixture model…
A weighted Gaussian approximation to tail product-limit process for Pareto-like distributions of randomly right-truncated data is provided and a new consistent and asymptotically normal estimator of the extreme value index is derived. A simulation study is carried out to evaluate the finite sample behavior of the proposed estimator.
In this paper, we prove $L_p,\ p\geq 2$ and almost sure convergence of tail index estimator mentioned in \cite{grama2008} under random censoring and several assumptions. $p$th moment of the error of the estimator is proved to be of order $O\left(\frac{1}{\log^{mκ/2}n}\right)$ with given assumptions. We also perform several finite sample simulations to quantify performance of this estimator. Finite sample results show that the proposed estimator is effective in finding underlying tail index even when censor rate is high.
A possibility to give strong mathematical definitions of outliers and heavy tailed distributions or their modification is discussed. Some alternatives for the notion of tail index are proposed. Key words: outliers, heavy tails, tail index.
Based on suitable left-truncated or censored data, two flexible classes of $M$-estimations of Weibull tail coefficient are proposed with two additional parameters bounding the impact of extreme contamination. Asymptotic normality with $\sqrt {n}$-rate of convergence is obtained. Its robustness is discussed via its asymptotic relative efficiency and influence function. It is further demonstrated by a small scale of simulations and an empirical study on CRIX.
