In this paper, we introduce reduced-bias estimators for the estimation of the tail index of a Pareto-type distribution. This is achieved through the use of a regularised weighted least squares with an exponential regression model for log-spacings of top order statistics. The asymptotic properties of the proposed estimators are investigated analytically and found to be asymptotically unbiased, consistent and normally distributed. Also, the finite sample behaviour of the estimators are studied through a simulations theory. The proposed estimators were found to yield low bias and MSE. In addition, the proposed estimators are illustrated through the estimation of the tail index of the underlying distribution of claims from the insurance industry.
The subject of tail estimation for randomly censored data from a heavy tailed distribution receives growing attention, motivated by applications for instance in actuarial statistics. The bias of the available estimators of the extreme value index can be substantial and depends strongly on the amount of censoring. We review the available estimators, propose a new bias reduced estimator, and show how shrinkage estimation can help to keep the MSE under control. A bootstrap algorithm is proposed to construct confidence intervals. We compare these new proposals with the existing estimators through …
We revisit multivariate extreme value theory modeling by emphasizing multivariate regular variations and the multivariate Breiman Lemma. This allows us to recover in a simple framework the most popular multivariate extreme value distributions, such as the logistic, negative logistic, Dirichlet, extremal-$t$ and Hüsler-Reiss models. In a second part of the paper, we focus on the Hüsler-Reiss Pareto model and its surprising exponential family property. After a thorough study of this exponential family structure, we focus on maximum likelihood estimation. We also consider the generalized Hüsler-R…
The Zenga (1984) inequality curve is constant in p for Type I Pareto distributions. This characterizing behavior will be exploited to obtain graphical and analytical tools for tail analysis and goodness of fit tests. A testing procedure for Pareto-type behavior based on a regression of technique will be introduced.
In recent years several attempts have been made to extend tail modelling towards the modal part of the data. Frigessi et al. (2002) introduced dynamic mixtures of two components with a weight function π = π(x) smoothly connecting the bulk and the tail of the distribution. Recently, Naveau et al. (2016) reviewed this topic, and, continuing on the work by Papastathopoulos and Tawn (2013), proposed a statistical model which is in compliance with extreme value theory and allows for a smooth transition between the modal and tail part. Incorporating second order rates of convergence for distribution…
In this paper, we define a kernel estimator for the tail index of a Pareto-type distribution under random right-truncation and establish its asymptotic normality. A simulation study shows that, compared to the estimators recently proposed by Gardes & Stupfler (2015) and Benchaira et al. (2015), this newly introduced estimator behaves better, in terms of bias and mean squared error, for small samples.
We study the problem of constructing honest and adaptive confidence intervals for the tail coefficient in the second order Pareto model, when the second order coefficient is unknown. This problem is translated into a testing problem on the second order parameter. By constructing an appropriate model and an associated test statistic, we provide a uniform and adaptive confidence interval for the first order parameter. We also provide an almost matching lower bound, which proves that the result is minimax optimal up to a logarithmic factor.
We introduce a kernel estimator, to the tail index of a right-censored Pareto-type distribution, that generalizes Worms's one (Worms and Worms, 2014)in terms of weight coefficients. Under some regularity conditions, the asymptotic normality of the proposed estimator is established. In the framework of the second-order condition, we derive an asymptotically bias-reduced version to the new estimator. Through a simulation study, we conclude that one of the main features of the proposed kernel estimator is its smoothness contrary to Worms's one, which behaves, rather erratically, as a function of …
The Generalized Pareto Distribution (GPD) plays a central role in modelling heavy tail phenomena in many applications. Applying the GPD to actual datasets however is a non-trivial task. One common way suggested in the literature to investigate the tail behaviour is to take logarithm to the original dataset in order to reduce the sample variability. Inspired by this, we propose and study the Exponentiated Generalized Pareto Distribution (exGPD), which is created via log-transform of the GPD variable. After introducing the exGPD we derive various distributional quantities, including the moment g…
Recently attention has been drawn to practical problems with the use of unbounded Pareto distributions, for instance when there are natural upper bounds that truncate the probability tail. Aban, Meerschaert and Panorska (2006) derived the maximum likelihood estimator for the Pareto tail index of a truncated Pareto distribution with a right truncation point $T$. The Hill (1975) estimator is then obtained by letting $T \to \infty$. The problem of extreme value estimation under right truncation was also introduced in Nuyts (2010) who proposed a similar estimator for the tail index and considered …
Estimating the tail index of Pareto-type distributions from geometric records
Universidad de Zaragoza（萨拉戈萨大学）；Instituto de Biocomputación y Física de Sistemas Complejos (BIFI), Universidad de Zaragoza（萨拉戈萨大学生物计算与复杂系统物理研究所 (BIFI)）；Universidad de Chile（智利大学）
In risk analysis, a global fit that appropriately captures the body and the tail of the distribution of losses is essential. Modelling the whole range of the losses using a standard distribution is usually very hard and often impossible due to the specific characteristics of the body and the tail of the loss distribution. A possible solution is to combine two distributions in a splicing model: a light-tailed distribution for the body which covers light and moderate losses, and a heavy-tailed distribution for the tail to capture large losses. We propose a splicing model with a mixed Erlang (ME)…
A variety of estimators for the parameters of the Generalized Pareto distribution, the approximating distribution for excesses over a high threshold, have been proposed, always assuming the underlying data to be independent. We recently proved that the likelihood moment estimators are consistent estimators for the parameters of the Generalized Pareto distribution for the case where the underlying data arises from a (stationary) linear process with heavy-tailed innovations. In this paper we derive the bivariate asymptotic normality under some additional assumptions and give an explicit example …
In this article we show the relationship between the Pareto distribution and the gamma distribution. This shows that the second one, appropriately extended, explains some anomalies that arise in the practical use of extreme value theory. The results are useful to certain phenomena that are fitted by the Pareto distribution but, at the same time, they present a deviation from this law for very large values. Two examples of data analysis with the new model are provided. The first one is on the influence of climate variability on the occurrence of tropical cyclones. The second one on the analysis…
The paper considers the problem of multi-objective decision support when outcomes are uncertain. We extend the concept of Pareto-efficient decisions to take into account the uncertainty of decision outcomes across varying contexts. This enables quantifying trade-offs between decisions in terms of tail outcomes that are relevant in safety-critical applications. We propose a method for learning efficient decisions with statistical confidence, building on results from the conformal prediction literature. The method adapts to weak or nonexistent context covariate overlap and its statistical guaran…
This article describes mathematical methods for estimating the top-tail of the wealth distribution and therefrom the share of total wealth that the richest $p$ percent hold, which is an intuitive measure of inequality. As the data base for the top-tail of the wealth distribution is inevitably less complete than the data for lower wealth, the top-tail distribution is replaced by a parametric model based on a Pareto distribution. The different methods for estimating the parameters are compared and new simulations are presented which favor the maximum-likelihood estimator for the Pareto parameter…
Here we suppose that the observed random variable has cumulative distribution function $F$ with regularly varying tail, i.e. $1-F \in RV_{-α}$, $α> 0$. Using the results about exponential order statistics we investigate logarithms of ratios of two order statistics of a sample of independent observations on Pareto distributed random variable with parameter $α$. Short explicit formulae for its mean and variance are obtained. Then we transform this function in such a way that to obtain unbiased, asymptotically efficient, and asymptotically normal estimator for $α$. Finally we simulate Pareto samp…
Estimation of tail quantities, such as expected shortfall or Value at Risk, is a difficult problem. We show how the theory of nonlinear expectations, in particular the Data-robust expectation introduced in [5], can assist in the quantification of statistical uncertainty for these problems. However, when we are in a heavy-tailed context (in particular when our data are described by a Pareto distribution, as is common in much of extreme value theory), the theory of [5] is insufficient, and requires an additional regularization step which we introduce. By asking whether this regularization is pos…
We present a nonparametric family of estimators for the tail index of a Pareto-type distribution when covariate information is available. Our estimators are based on a weighted sum of the log-spacings between some selected observations. This selection is achieved through a moving window approach on the covariate domain and a random threshold on the variable of interest. Asymptotic normality is proved under mild regularity conditions and illustrated for some weight functions. Finite sample performances are presented on a real data study.
In this paper, we discuss a method to define prior distributions for the threshold of a generalised Pareto distribution, in particular when its applications are directed to heavy-tailed data. We propose to assign prior probabilities to the order statistics of a given set of observations. In other words, we assume that the threshold coincides to one of the data points. We show two ways of defining a prior: by assigning equal mass to each order statistic, that is a uniform prior, and by considering the worth that every order statistic has in representing the true threshold. Both proposed priors …
After Pareto distribution has been validated for sea clutter returns in varied scenarios, some heuristics of adaptive-thresholding appeared in the literature for constant false alarm rate (CFAR) criteria. These schemes used the same adaptive-thresholding form that was originally derived for detecting Swerling-I (exponential) target in exponentially distributed clutter. Statistical procedures obtained under such idealistic assumptions would affect the detection performance when applied to newer target and clutter models, esp. heavy tail distributions like Pareto. Further, in addition to the sea…
This paper deals with optimally-robust parameter estimation in generalized Pareto distributions (GPDs). These arise naturally in many situations where one is interested in the behavior of extreme events as motivated by the Pickands-Balkema-de Haan extreme value theorem (PBHT). The application we have in mind is calculation of the regulatory capital required by Basel II for a bank to cover operational risk. In this context the tail behavior of the underlying distribution is crucial. This is where extreme value theory enters, suggesting to estimate these high quantiles parameterically using, e.g…
A multivariate distribution function F is in the max-domain of attraction of an extreme value distribution if and only if this is true for the copula corresponding to F and its univariate margins. Aulbach et al. (2012a) have shown that a copula satisfies the extreme value condition if and only if the copula is tail equivalent to a generalized Pareto copula (GPC). In this paper we propose a chi-square goodness-of-fit test in arbitrary dimension for testing whether a copula is in a certain neighborhood of a GPC. The test can be applied to stochastic processes as well to check whether the corresp…
We propose a class of weighted least squares estimators for the tail index of a distribution function with a regularly varying upper tail. Our approach is based on the method developed by \cite{Holan2010} for the Parzen tail index. Asymptotic normality of the estimators is proved. Through a simulation study, these and earlier estimators are compared in the Pareto and Hall models using the mean squared error as criterion. The results show that the weighted least squares estimator is better than the other estimators investigated.
We analyze the \textit{Large Deviation Probability (LDP)} of linear factor models generated from non-identically distributed components with \textit{regularly-varying} tails, a large subclass of heavy tailed distributions. An efficient sampling method for LDP estimation of this class is introduced and theoretically shown to exponentially outperform the crude Monte-Carlo estimator, in terms of the coverage probability and the confidence interval's length. The theoretical results are empirically validated through stochastic simulations on independent non-identically Pareto distributed factors. T…
By means of a Lynden-Bell integral with deterministic threshold, Worms and Worms [A Lynden-Bell integral estimator for extremes of randomly truncated data. Statist. Probab. Lett. 2016; 109: 106-117] recently introduced an asymptotically normal estimator of the tail index for randomly right-truncated Pareto-type data. In this context, we consider the random threshold case to derive a Hill-type estimator and establish its consistency and asymptotic normality. A simulation study is carried out to evaluate the finite sample behavior of the proposed estimator.
It was shown that when one disposes of a parametric information of the truncation distribution, the semiparametric estimator of the distribution function for truncated data (Wang, 1989) is more efficient than the nonparametric one. On the basis of this estimation method, we derive an estimator for the tail index of Pareto-type distributions that are randomly right-truncated and establish its consistency and asymptotic normality. The finite sample behavior of the proposed estimator is carried out by simulation study. We point out that, in terms of both bias and root of the mean squared error, o…
De Haan and Pereira (2006) provided models for spatial extremes in the case of stationarity, which depend on just one parameter β > 0 measuring tail dependence, and they proposed different estimators for this parameter. This framework was supplemented in Falk (2011) by establishing local asymptotic normality (LAN) of a corresponding point process of exceedances above a high multivariate threshold, yielding in particular asymptotic efficient estimators. The estimators investigated in these papers are based on a finite set of points t1,...,td, at which observations are taken. We generalize this …
This paper considers Bayesian multiple testing under sparsity for polynomial-tailed distributions satisfying a monotone likelihood ratio property. Included in this class of distributions are the Student's t, the Pareto, and many other distributions. We prove some general asymptotic optimality results under fixed and random thresholding. As examples of these general results, we establish the Bayesian asymptotic optimality of several multiple testing procedures in the literature for appropriately chosen false discovery rate levels. We also show by simulation that the Benjamini-Hochberg procedure…
The most popular approach in extreme value statistics is the modelling of threshold exceedances using the asymptotically motivated generalised Pareto distribution. This approach involves the selection of a high threshold above which the model fits the data well. Sometimes, few observations of a measurement process might be recorded in applications and so selecting a high quantile of the sample as the threshold leads to almost no exceedances. In this paper we propose extensions of the generalised Pareto distribution that incorporate an additional shape parameter while keeping the tail behaviour…
This article is devoted to the study of tail index estimation based on i.i.d. multivariate observations, drawn from a standard heavy-tailed distribution, i.e. of which 1-d Pareto-like marginals share the same tail index. A multivariate Central Limit Theorem for a random vector, whose components correspond to (possibly dependent) Hill estimators of the common shape index alpha, is established under mild conditions. Motivated by the statistical analysis of extremal spatial data in particular, we introduce the concept of (standard) heavy-tailed random field of tail index alpha and show how this l…
The upper tail of a claim size distribution of a property line of business is frequently modelled by Pareto distribution. However, the upper tail does not need to be Pareto distributed, extraordinary shapes are possible. Here, the opportunities for the modelling of loss distributions are extended. The basic idea is the adjustment of a base distribution for their tails. The (generalised) Pareto distribution is used as base distribution for different reasons. The upper tail is in the focus and can be modelled well for special cases by a discrete mixture of the base distribution with a combinatio…
In this paper, we propose an estimator of the second-order parameter of randomly right-truncated Pareto-type distributions data and establish its consistency and asymptotic normality. Moreover, we derive an asymptotically unbiased estimator of the tail index and study its asymptotic behaviour. Our considerations are based on a useful Gaussian approximation of the tail product-limit process recently given by Benchaira et al. [Tail product-limit process for truncated data with application to extreme value index estimation. Extremes, 2016; 19: 219-251] and the results of Gomes et al. [Semi-parame…
A coupling method is developed for univariate extreme value theory , providing an alternative to the use of the tail empirical/quantile processes. Emphasizing the Peak-over-Threshold approach that approximates the distribution above high threshold by the Generalized Pareto distribution, we compare the empirical distribution of exceedances and the empirical distribution associated to the limit Generalized Pareto model and provide sharp bounds for their Wasser-stein distance in the second order Wasserstein space. As an application , we recover standard results on the asymptotic behavior of the H…
A tail empirical process for heavy-tailed and right-censored data is introduced and its Gaussian approximation is established. In this context, a (weighted) new Hill-type estimator for positive extreme value index is proposed and its consistency and asymptotic normality are proved by means of the aforementioned process in the framework of second-order conditions of regular variation. In a comparative simulation study, the newly defined estimator is seen to perform better than the already existing ones in terms of both bias and mean squared error. As a real data example, we apply our estimation…
We consider the problem of estimating the tail index $α$ of a distribution satisfying a $(α, β)$ second-order Pareto-type condition, where βis the second-order coefficient. When $β$ is available, it was previously proved that $α$ can be estimated with the oracle rate $n^{-β/(2β+1)}$. On the contrary, when $β$ is not available, estimating $α$ with the oracle rate is challenging; so additional assumptions that imply the estimability of $β$ are usually made. In this paper, we propose an adaptive estimator of $α$, and show that this estimator attains the rate $(n/\log\log n)^{-β/(2β+1)}$ without a…
Let $X_1, \ldots, X_n$ be some i.i.d. observations from a heavy tailed distribution $F$, i.e. such that the common distribution of the excesses over a high threshold $u_n$ can be approximated by a Generalized Pareto Distribution $G_{γ,σ_n}$ with $γ>0$. This work is devoted to the problem of finding confidence regions for the couple $(γ,σ_n)$ : combining the empirical likelihood methodology with estimation equations (close but not identical to the likelihood equations) introduced by J. Zhang (Australian and New Zealand J. Stat n.49(1), 2007), asymptotically valid confidence regions for $(γ,σ_n)…
Peaks-over-threshold analysis using the generalized Pareto distribution is widely applied in modelling tails of univariate random variables, but much information may be lost when complex extreme events are studied using univariate results. In this paper, we extend peaks-over-threshold analysis to extremes of functional data. Threshold exceedances defined using a functional $r$ are modelled by the generalized $r$-Pareto process, a functional generalization of the generalized Pareto distribution that covers the three classical regimes for the decay of tail probabilities, and that is the only pos…
We prove the consistency of the Power-Law Fit PLFit method proposed by Clauset et al.(2009) to estimate the power-law exponent in data coming from a distribution function with regularly-varying tail. In the complex systems community, PLFit has emerged as the method of choice to estimate the power-law exponent. Yet, its mathematical properties are still poorly understood. The difficulty in PLFit is that it is a minimum-distance estimator. It first chooses a threshold that minimizes the Kolmogorov-Smirnov distance between the data points larger than the threshold and the Pareto tail, and then ap…
The Zenga (1984) inequality curve is constant in p for Type I Pareto distributions. We show that this property holds exactly only for the Pareto distribution and, asymptotically, for distributions with power tail with index -a, with a greater than 1. Exploiting these properties one can develop powerful tools to analyze and estimate the tail of a distribution.
For extreme value estimation we propose to use a model with a Dirichlet process mixture of gamma densities in the center and generalized Pareto densities for the tails. Due to the randomness in the center and a heavy tailed density in the tails density estimation and posterior inference for high quantiles are possible. The approach can be used in a "default" manner on the positive reals because it works when prior information is unavailable. The proposed model can be easy to implement and a sensitivity analysis is provided. We applied the proposed model for simulated and real data sets.
The Pareto model is very popular in risk management, since simple analytical formulas can be derived for financial downside risk measures (Value-at-Risk, Expected Shortfall) or reinsurance premiums and related quantities (Large Claim Index, Return Period). Nevertheless, in practice, distributions are (strictly) Pareto only in the tails, above (possible very) large threshold. Therefore, it could be interesting to take into account second order behavior to provide a better fit. In this article, we present how to go from a strict Pareto model to Pareto-type distributions. We discuss inference, an…
In this paper we obtain closed expressions for the probability distribution function, when we consider aggregated risks with multivariate dependent Pareto distributions. We work with the dependent multivariate Pareto type II proposed by Arnold (1983, 2015), which is widely used in insurance and risk analysis. We begin with the individual risk model, where we obtain the probability density function (PDF), which corresponds to a second kind beta distribution. We obtain several risk measures including the VaR, TVaR and other tail measures. Then, we consider collective risk model based on dependen…
Power law distributions, in particular Pareto distributions, describe data across diverse areas of study. We have developed a package in R to estimate the tail index for such datasets focusing on speed (in particular with large datasets), keeping in mind ease of use, as well as accuracy. In this document, we provide a user guide to our package along with the results obtained highlighting the speed advantages of our package.
