We introduce an alternative to the notion of `fast rate' in Learning Theory, which coincides with the optimal error rate when the given class happens to be convex and regular in some sense. While it is well known that such a rate cannot always be attained by a learning procedure (i.e., a procedure that selects a function in the given class), we introduce an aggregation procedure that attains that rate under rather minimal assumptions -- for example, that the $L_q$ and $L_2$ norms are equivalent on the linear span of the class for some $q>2$, and the target random variable is square-integrable.
In this work, we establish risk bounds for the Empirical Risk Minimization (ERM) with both dependent and heavy-tailed data-generating processes. We do so by extending the seminal works of Mendelson [Men15, Men18] on the analysis of ERM with heavy-tailed but independent and identically distributed observations, to the strictly stationary exponentially $β$-mixing case. Our analysis is based on explicitly controlling the multiplier process arising from the interaction between the noise and the function evaluations on inputs. It allows for the interaction to be even polynomially heavy-tailed, whic…
We consider the fitting of heavy tailed data and distribution with a special attention to distributions with a non--standard shape in the "body" of the distribution. To this end we consider a dense class of heavy tailed distributions introduced recently, employing an EM algorithm for the the maximum likelihood estimates of its parameters. We present methods for fitting to observed data, histograms, censored data, as well as to theoretical distributions. Numerical examples are provided with simulated data and a benchmark reinsurance dataset. We empirically demonstrate that our model can provide…
We analyze the oracle complexity of sampling from polynomially decaying heavy-tailed target densities based on running the Unadjusted Langevin Algorithm on certain transformed versions of the target density. The specific class of closed-form transformation maps that we construct are shown to be diffeomorphisms, and are particularly suited for developing efficient diffusion-based samplers. We characterize the precise class of heavy-tailed densities for which polynomial-order oracle complexities (in dimension and inverse target accuracy) could be obtained, and provide illustrative examples. We h…
Pompeu Fabra University（庞培法布拉大学）；ICREA（加泰罗尼亚研究与高等研究院）；Barcelona Graduate School of Economics（巴塞罗那经济研究生院）；The Australian National University（澳大利亚国立大学）；Sorbonne University（索邦大学）
We survey some of the recent advances in mean estimation and regression function estimation. In particular, we describe sub-Gaussian mean estimators for possibly heavy-tailed data both in the univariate and multivariate settings. We focus on estimators based on median-of-means techniques but other methods such as the trimmed mean and Catoni's estimator are also reviewed. We give detailed proofs for the cornerstone results. We dedicate a section on statistical learning problems--in particular, regression function estimation--in the presence of possibly heavy-tailed data.
We make use of the empirical process theory to approximate the adapted Hill estimator, for censored data, in terms of Gaussian processes. Then, we derive its asymptotic normality, only under the usual second-order condition of regular variation. Our methodology allows to relax the assumptions, made in Einmahl, Fils-Villetard and Guillou(2008), on the heavy-tailed distribution functions and the sample fraction of upper order statistics.
Several convenient methods for calculation of fractional absolute moments are given with application to heavy tailed distributions. We use techniques of fractional differentiation to obtain formulae for $E[|X-μ|^γ]$ with $1<γ<2$ and $μ\in\mathbb{R}$. The main focus is on heavy tailed distributions, several examples are given with analytical expressions of fractional absolute moments. As applications, we calculate the fractional moment errors for both prediction and parameter estimation problems.
The central limit theorem introduced by Stute [The central limit theorem under random censorship. Ann. Statist. 1995; 23: 422-439] does not hold for some class of heavy-tailed distributions. In this paper, we make use of the extreme value theory to propose an alternative estimating approach of the mean ensuring the asymptotic normality property. A simulation study is carried out to evaluate the performance of this estimation procedure
We show that for local alternatives which are not square integrable the intermediate (or Kallenberg) efficiency of the Neyman-Pearson test for uniformity with respect to the classical Kolmogorov-Smirnov test is equal to infinity. Contrary to this, for local square integrable alternatives the intermediate efficiency is finite and can be explicitly calculated.
It has been shown that some macroeconomic time series, especially those where outliers could be present, can be well modelled using heavy tailed distributions for the noise components. Methods for deciding when and where heavy-tailed models should be preferred are investigated. These investigations primarily focus on automatic methods for model identification and selection. Current methods are extended to incorporate a non-Gaussian selection element, and various different criteria for deciding on which overall model should be used are examined.
Gini-type correlation coefficients have become increasingly important in a variety of research areas, including economics, insurance and finance, where modelling with heavy-tailed distributions is of pivotal importance. In such situations, naturally, the classical Pearson correlation coefficient is of little use. On the other hand, it has been observed that when light-tailed situations are of interest, and hence when both the Gini-type and Pearson correlation coefficients are well-defined and finite, then these coefficients are related and sometimes even coincide. In general, understanding how…
We study high-dimensional signal recovery from non-linear measurements with design vectors having elliptically symmetric distribution. Special attention is devoted to the situation when the unknown signal belongs to a set of low statistical complexity, while both the measurements and the design vectors are heavy-tailed. We propose and analyze a new estimator that adapts to the structure of the problem, while being robust both to the possible model misspecification characterized by arbitrary non-linearity of the measurements as well as to data corruption modeled by the heavy-tailed distribution…
In this paper, we investigate the extreme-value methodology, to propose an improved estimator of the conditional tail expectation ($CTE$) for a loss distribution with a finite mean but infinite variance. The present work introduces a new estimator of the $CTE$ based on the bias-reduced estimators of high quantile for heavy-tailed distributions. The asymptotic normality of the proposed estimator is established and checked, in a simulation study. Moreover, we compare, in terms of bias and mean squared error, our estimator with the known old estimator.
Estimating high quantiles plays an important role in the context of risk management. This involves extrapolation of an unknown distribution function. In this paper we propose three methods, namely, the normal approximation method, the likelihood ratio method and the data tilting method, to construct confidence regions for high quantiles of a heavy tailed distribution. A simulation study prefers the data tilting method.
Empirical-likelihood-based confidence intervals for a mean were introduced by Owen [Biometrika 75 (1988) 237-249], where at least a finite second moment is required. This excludes some important distributions, for example, those in the domain of attraction of a stable law with index between 1 and 2. In this article we use a method similar to Qin and Wong [Scand. J. Statist. 23 (1996) 209-219] to derive an empirical-likelihood-based confidence interval for the mean when the underlying distribution has heavy tails. Our method can easily be extended to obtain a confidence interval for any order o…
We consider strictly stationary heavy tailed time series whose finite-dimensional exponent measures are concentrated on axes, and hence their extremal properties cannot be tackled using classical multivariate regular variation that is suitable for time series with extremal dependence. We recover relevant information about limiting behavior of time series with extremal independence by introducing a sequence of scaling functions and conditional scaling exponent. Both quantities provide more information about joint extremes than a widely used tail dependence coefficient. We calculate the scaling …
The paper introduces structured machine learning regressions for heavy-tailed dependent panel data potentially sampled at different frequencies. We focus on the sparse-group LASSO regularization. This type of regularization can take advantage of the mixed frequency time series panel data structures and improve the quality of the estimates. We obtain oracle inequalities for the pooled and fixed effects sparse-group LASSO panel data estimators recognizing that financial and economic data can have fat tails. To that end, we leverage on a new Fuk-Nagaev concentration inequality for panel data cons…
We use the so-called t-Hill tail index estimator proposed by Fabián(2001), rather than Hill's one, to derive a robust estimator for the distortion risk premium of loss. Under the second-order condition of regular variation, we establish its asymptotic normality. By simulation study, we show that this new estimator is more robust than of Necir and Meraghni 2009 both for small and large samples.
We consider stationary time series $\{X_j, j \in Z\} whose finite dimensional distributions are regularly varying with extremal independence. We assume that for each $h \geq 1$, conditionally on $X_0$ to exceed a threshold tending to infinity, the conditional distribution of $X_h$ suitably normalized converges weakly to a non degenerate distribution. We consider in this paper the estimation of the normalization and of the limiting distribution.
This article deals with the hypothesis test for the extremely heavy-tailed distributions with infinite mean or variance by using a truncated sample mean. We obtain three necessary and sufficient conditions under which the asymptotic distribution of the truncated test statistics converges to normal, neither normal nor stable or converges to $-\infty$ or the combination of stable distributions, respectively. The numerical simulation illustrates an application of the theoretical results above in the hypothesis testing.
We propose and analyze a new estimator of the covariance matrix that admits strong theoretical guarantees under weak assumptions on the underlying distribution, such as existence of moments of only low order. While estimation of covariance matrices corresponding to sub-Gaussian distributions is well-understood, much less in known in the case of heavy-tailed data. As K. Balasubramanian and M. Yuan write, "data from real-world experiments oftentimes tend to be corrupted with outliers and/or exhibit heavy tails. In such cases, it is not clear that those covariance matrix estimators .. remain opti…
In this work, we focus on some conditional extreme risk measures estimation for elliptical random vectors. In a previous paper, we proposed a methodology to approximate extreme quantiles, based on two extremal parameters. We thus propose some estimators for these parameters, and study their asymptotic properties in the case of heavy-tailed distributions. Thereafter, from these parameters, we construct extreme conditional quantiles estimators, and give their consistency properties. Using recent results on the asymptotic relationship between quantiles and other risk measures, we deduce estimator…
The Spurious Factor Dilemma: Robust Inference in Heavy-Tailed Elliptical Factor Models
KLASMOE and School of Mathematics and Statistics, Northeast Normal University（东北师范大学数学与统计学院）；Department of Statistics and Data Science, National University of Singapore（新加坡国立大学统计与数据科学系）；Department of Mathematics, Shanghai University（上海大学数学系）
Privacy of SGD under Gaussian or Heavy-Tailed Noise: Guarantees without Gradient Clipping
Inria, CNRS, Ecole Normale Supérieure PSL Research University（法国国家科学研究中心、巴黎高等师范学院PSL研究大学）；Department of Management Science and Information Systems（管理科学与信息系统系）；Faculty of Engineering and Natural Sciences（工程与自然科学学院）；Department of Mathematics（数学系）
We explore the Cauchy and a new heavy tailed (Fuquene, Perez and Pericchi (2011)) priors to estimate proportions on small areas. Hierarchical models and the Binomial likelihood in the exponential family form are used. We believe that the heavy tailed priors in survey sampling settings could be more effective than the choice of noninformative priors to eliminate antipathy towards methods that involve subjective elements or assumptions. To illustrate the robust Bayesian approach, we apply this methodology in a popular example: "the clement problem". Finally, we recommend to use the Cauchy prior …
University of Wisconsin-Madison（威斯康星大学麦迪逊分校）；University of California, San Diego（加州大学圣地亚哥分校）；Simons Institute for the Theory of Computing（西蒙斯计算理论研究所）
We study the fundamental task of outlier-robust mean estimation for heavy-tailed distributions in the presence of sparsity. Specifically, given a small number of corrupted samples from a high-dimensional heavy-tailed distribution whose mean $μ$ is guaranteed to be sparse, the goal is to efficiently compute a hypothesis that accurately approximates $μ$ with high probability. Prior work had obtained efficient algorithms for robust sparse mean estimation of light-tailed distributions. In this work, we give the first sample-efficient and polynomial-time robust sparse mean estimator for heavy-taile…
Locally adaptive shrinkage in the Bayesian framework is achieved through the use of local-global prior distributions that model both the global level of sparsity as well as individual shrinkage parameters for mean structure parameters. The most popular of these models is the Horseshoe prior and its variants due to their spike and slab behavior involving an asymptote at the origin and heavy tails. In this article, we present an alternative Horseshoe prior that exhibits both a sharper asymptote at the origin as well as heavier tails, which we call the Heavy-tailed Horseshoe prior. We prove that …
This article studies the convergence rate of the sample mean for $φ$-mixing dependent random variables with finite means and infinite variances. Dividing the sample mean into sum of the average of the main parts and the average of the tailed parts, we not only obtain the convergence rate of the sample mean but also prove that the convergence rate of the average of the main parts is faster than that of the average of the tailed parts.
A novel statistical method is proposed and investigated for estimating a heavy tailed density under mild smoothness assumptions. Statistical analyses of heavy-tailed distributions are susceptible to the problem of sparse information in the tail of the distribution getting washed away by unrelated features of a hefty bulk. The proposed Bayesian method avoids this problem by incorporating smoothness and tail regularization through a carefully specified semiparametric prior distribution, and is able to consistently estimate both the density function and its tail index at near minimax optimal rate…
A variety of estimators for the parameters of the Generalized Pareto distribution, the approximating distribution for excesses over a high threshold, have been proposed, always assuming the underlying data to be independent. We recently proved that the likelihood moment estimators are consistent estimators for the parameters of the Generalized Pareto distribution for the case where the underlying data arises from a (stationary) linear process with heavy-tailed innovations. In this paper we derive the bivariate asymptotic normality under some additional assumptions and give an explicit example …
This paper deals with the extreme value analysis for the triangular arrays, which appear when some parameters of the mixture model vary as the number of observations grow. When the mixing parameter is small, it is natural to associate one of the components with "an impurity" (in case of regularly varying distribution, "heavy-tailed impurity"), which "pollutes" another component. We show that the set of possible limit distributions is much more diverse than in the classical Fisher-Tippett-Gnedenko theorem, and provide the numerical examples showing the efficiency of the proposed model for study…
We study a new random matrix ensemble $X$ which is constructed by an application of a two dimensional linear filter to a matrix of iid random variables with infinite fourth moments. Our result gives asymptotic lower and upper bounds for the spectral norm of the (centered) sample covariance matrix $XX^\T$ when the number of columns as well es the number of rows of $X$ tend to infinity.
Expectiles define the only law-invariant, coherent and elicitable risk measure apart from the expectation. The popularity of expectile-based risk measures is steadily growing and their properties have been studied for independent data, but further results are needed to use extreme expectiles with dependent time series such as financial data. In this paper we establish a basis for inference on extreme expectiles and expectile-based marginal expected shortfall in a general $β$-mixing context that encompasses ARMA, ARCH and GARCH models with heavy-tailed innovations. Simulations and applications …
We use bias-reduced estimators of high quantiles, of heavy-tailed distributions, to introduce a new estimator of the mean in the case of infinite second moment. The asymptotic normality of the proposed estimator is established and checked, in a simulation study, by four of the most popular goodness-of-fit tests for different sample sizes. Moreover, we compare, in terms of bias and mean squared error, our estimator with Peng's estimator (Peng, 2001) and we evaluate the accuracy of some resulting confidence intervals.
We study heavy-tailed Hermitian random matrices that are unitarily invariant. The invariance implies that the eigenvalue and eigenvector statistics are decoupled. The motivating question has been whether a freely stable random matrix has stable eigenvalue statistics for the largest eigenvalues in the tail. We investigate this question through the use of both numerical and analytical means, the latter of which makes use of the supersymmetry method. A surprising behaviour is uncovered in that a freely stable random matrix does not necessarily yield stable statistics and if it does then it might …
