This paper investigates the decay rate of the probability that the row sum of a triangular array of truncated heavy tailed random variables is larger than an integer (k) times the truncating threshold, as both - the number of summands and the threshold go to infinity. The method of attack for this problem is significantly different from the one where k is not an integer, and requires much sharper estimates.
In this paper the question of the extent to which truncated heavy tailed random vectors, taking values in a Banach space, retain the characteristic features of heavy tailed random vectors, is answered from the point of view of the central limit theorem.
It is well-known that large deviations of random walks driven by independent and identically distributed heavy-tailed random variables are governed by the so-called principle of one large jump. We note that further subtleties hold for such random walks in the large deviation scale which we call hidden large deviation. We apply this idea in the context of queueing processes with heavy-tailed service times and study approximations of severe congestion times for (buffered) queues. We conclude with simulated examples to verify our results.
It is known that large deviations of sums of subexponential random variables are most likely realised by deviations of a single random variable. In this article we give a detailed picture of how subexponential random variables are distributed when a large deviation of their sum is observed.
We prove large deviation results for sums of heavy-tailed random elements in rather general convex cones being semigroups equipped with a rescaling operation by positive real numbers. In difference to previous results for the cone of convex sets, our technique does not use the embedding of cones in linear spaces. Examples include the cone of convex sets with the Minkowski addition, positive half-line with maximum operation and the family of square integrable functions with arithmetic addition and argument rescaling.
A large deviations principle is established for the joint law of the empirical measure and the flow measure of a renewal Markov process on a finite graph. We do not assume any bound on the arrival times, allowing heavy tailed distributions. In particular, the rate functional is in general degenerate (it has a nontrivial set of zeros) and not strictly convex. These features show a behavior highly different from what one may guess with a heuristic Donsker-Varadhan analysis of the problem.
This paper studies the effect of truncation on the large deviations behavior of the partial sum of a triangular array coming from a truncated power law model. Each row of the triangular array consists of i.i.d. random vectors, whose distribution matches a power law on a ball of radius going to infinity, and outside that it has a light-tailed modification. The random vectors are assumed to be R^d-valued. It turns out that there are two regimes depending on the growth rate of the truncating threshold, so that in one regime, much of the heavy tailedness is retained, while in the other regime, the…
We study large deviation probabilities for a sum of dependent random variables from a heavy-tailed factor model, assuming that the components are regularly varying. We identify conditions where both the factor and the idiosyncratic terms contribute to the behaviour of the tail-probability of the sum. A simple conditional Monte Carlo algorithm is also provided together with a comparison between the simulations and the large deviation approximation. We also study large deviation probabilities for stochastic processes with factor structure. The processes involved are assumed to be Levy processes …
Northwestern University（西北大学）；University of Washington（华盛顿大学）
The empirical success of deep learning is often attributed to SGD's mysterious ability to avoid sharp local minima in the loss landscape, as sharp minima are known to lead to poor generalization. Recently, empirical evidence of heavy-tailed gradient noise was reported in many deep learning tasks, and it was shown in Şimşekli (2019a,b) that SGD can escape sharp local minima under the presence of such heavy-tailed gradient noise, providing a partial solution to the mystery. In this work, we analyze a popular variant of SGD where gradients are truncated above a fixed threshold. We show that it ac…
Modern risk modelling approaches deal with vectors of multiple components. The components could be, for example, returns of financial instruments or losses within an insurance portfolio concerning different lines of business. One of the main problems is to decide if there is any type of dependence between the components of the vector and, if so, what type of dependence structure should be used for accurate modelling. We study a class of heavy-tailed multivariate random vectors under a non-parametric shape constraint on the tail decay rate. This class contains, for instance, elliptical distribu…
Let $U_m$ be an $m \times m$ Haar unitary matrix and $U_{[m,n]}$ be its $n \times n$ truncation. In this paper the large deviation is proven for the empirical eigenvalue density of $U_{[m,n]}$ as $m/n \to λ$ and $n \to \infty$. The rate function and the limit distribution are given explicitly. $U_{[m,n]}$ is the random matrix model of $quq$, where $u$ is a Haar unitary in a finite von Neumann algebra, $q$ is a certain projection and they are free. The limit distribution coincides with the Brown measure of the operator $quq$.
We consider the model of hashing with linear probing and we establish the moderate and large deviations for the total displacement in sparse tables. In this context, Weibull-like-tailed random variables appear. Deviations for sums of such heavy-tailed random variables are studied in \cite{Nagaev69-1,Nagaev69-2}. Here we adapt the proofs therein to deal with conditioned sums of such variables and solve the open question in \cite{TFC12}. By the way, we establish the deviations of the total displacement in full tables, which can be derived from the deviations of empirical processes of i.i.d.\ ran…
We address the important question of the extent to which random variables and vectors with truncated power tails retain the characteristic features of random variables and vectors with power tails. We define two truncation regimes, soft truncation regime and hard truncation regime, and show that, in the soft truncation regime, truncated power tails behave, in important respects, as if no truncation took place. On the other hand, in the hard truncation regime much of "heavy tailedness" is lost. We show how to estimate consistently the tail exponent when the tails are truncated, and suggest stat…
We prove that the largest and smallest root in modulus of random Kac polynomials have a non-universal behavior. They do not converge towards the edge of the support of the limiting distribution of the zeros. This non-universality is surprising as the large deviation principle for the empirical measure is universal. This is in sharp contrast with random matrix theory where the large deviation principle is non-universal but the fluctuations of the largest eigenvalue are universal. We show that the modulus of the largest zero is heavy tailed, with a number of finite moments bounded from above by …
We consider the FCFS G/G/n queue in the Halfin-Whitt regime, in the presence of heavy-tailed distributions (i.e. infinite variance). We prove that under minimal assumptions, i.e. only that processing times have finite 1 + epsilon moment and inter-arrival times have finite second moment, the sequence of stationary queue length distributions, normalized by $n^{\frac{1}{2}}$, is tight. All previous tightness results for the stationary queue length required that processing times have finite 2 + epsilon moment. Furthermore, we develop simple and explicit bounds on the stationary queue length in tha…
We extend classical results by A. V. Nagaev [Izv. Akad. Nauk UzSSR Ser. Fiz.--Mat. Nauk 6 (1969) 17--22, Theory Probab. Appl. 14 (1969) 51--64, 193--208] on large deviations for sums of i.i.d. regularly varying random variables to partial sum processes of i.i.d. regularly varying vectors. The results are stated in terms of a heavy-tailed large deviation principle on the space of càdlàg functions. We illustrate how these results can be applied to functionals of the partial sum process, including ruin probabilities for multivariate random walks and long strange segments. These results make preci…
We investigate how large deviations events cluster in the framework of an infinite moving average process with light-tailed noise and long memory. The long memory makes clusters larger, and the asymptotic behaviour of the size of the cluster turns out to be described by the first hitting time of a randomly shifted fractional Brownian motion with drift.
We study deviation of U-statistics when samples have heavy-tailed distribution so the kernel of the U-statistic does not have bounded exponential moments at any positive point. We obtain an exponential upper bound for the tail of the U-statistics which clearly denotes two regions of tail decay, the first is a Gaussian decay and the second behaves like the tail of the kernel. For several common U-statistics, we also show the upper bound has the right rate of decay as well as sharp constants by obtaining rough logarithmic limits which in turn can be used to develop LDP for U-statistics. In spite…
We examine the non-exit probability of a multidimensional Brownian motion from a growing truncated Weyl chamber. Different regimes are identified according to the growth speed, ranging from polynomial decay over stretched-exponential to exponential decay. Furthermore we derive associated large deviation principles for the empirical measure of the properly rescaled and transformed Brownian motion as the dimension grows to infinity. Our main tool is an explicit eigenvalue expansion for the transition probabilities before exiting the truncated Weyl chamber.
We consider a multivariate heavy-tailed stochastic volatility model and analyze the large-sample behavior of its sample covariance matrix. We study the limiting behavior of its entries in the infinite-variance case and derive results for the ordered eigenvalues and corresponding eigenvectors. Essentially, we consider two different cases where the tail behavior either stems from the i.i.d. innovations of the process or from its volatility sequence. In both cases, we make use of a large deviations technique for regularly varying time series to derive multivariate $α$-stable limit distributions o…
We investigate the probability that an insurance portfolio gets ruined within a finite time period under the assumption that the r largest claims are (partly) reinsured. We show that for regularly varying claim sizes the probability of ruin after reinsurance is also regularly varying in terms of the initial capital, and derive an explicit asymptotic expression for the latter. We establish this result by leveraging recent developments on sample-path large deviations for heavy tails. Our results allow, on the asymptotic level, for an explicit comparison between two well-known large-claim reinsur…
We study sample-path large deviations for Lévy processes and random walks with heavy-tailed jump-size distributions that are of Weibull type. Our main results include an extended form of an LDP (large deviations principle) in the $J_1$ topology, and a full LDP in the $M_1'$ topology. The rate function can be represented as the solution to a quasi-variational problem. The sharpness and applicability of these results are illustrated by a counterexample proving the nonexistence of a full LDP in the $J_1$ topology, and by an application to a first passage problem.
We consider a GI/GI/1 queue with the shortest remaining processing time discipline (SRPT) and light-tailed service times. Our interest is focused on the tail behavior of the sojourn-time distribution. We obtain a general expression for its large-deviations decay rate. The value of this decay rate critically depends on whether there is mass in the endpoint of the service-time distribution or not. An auxiliary priority queue, for which we obtain some new results, plays an important role in our analysis. We apply our SRPT-results to compare SRPT with FIFO from a large-deviations point of view.
We obtain concentration and large deviation for the sums of independent and identically distributed random variables with heavy-tailed distributions. Our concentration results are concerned with random variables whose distributions satisfy $\mathbb{P}(X>t) \leq {\rm e}^{- I(t)}$, where $I: \mathbb{R} \rightarrow \mathbb{R}$ is an increasing function and $I(t)/t \rightarrow α\in [0, \infty)$ as $t \rightarrow \infty$. Our main theorem can not only recover some of the existing results, such as the concentration of the sum of subWeibull random variables, but it can also produce new results for th…
We study asymptotic properties of spatially non-homogeneous random walks with non-integrable increments, including transience, almost-sure bounds, and existence and non-existence of moments for first-passage and last-exit times. In our proofs we also make use of estimates for hitting probabilities and large deviations bounds. Our results are more general than existing results in the literature, which consider only the case of sums of independent (typically, identically distributed) random variables. We do not assume the Markov property. Existing results that we generalize include a circle of i…
Large Deviation Upper Bounds and Improved MSE Rates of Nonlinear SGD: Heavy-tailed Noise and Power of Symmetry
Carnegie Mellon University（卡内基梅隆大学）；Faculty of Technical Sciences, University of Novi Sad（诺维萨德大学技术科学学院）；Faculty of Sciences, University of Novi Sad（诺维萨德大学理学院）
In this paper we study the asymptotic decay of finite time ruin probabilities for an insurance company that faces heavy-tailed claims, uses predictable investment strategies and makes investments in risky assets whose prices evolve according to quite general semimartingales. We show that the ruin problem corresponds to determining hitting probabilities for the solution to a randomly perturbed stochastic integral equation. We derive a large deviation result for the hitting probabilities that holds uniformly over a family of semimartingales and show that this result gives the asymptotic decay of…
We prove the convergence of the empirical spectral measure of Wishart matrices with size-dependent entries and characterize the limiting law by its moments. We apply our result to the cases where the entries are Bernoulli variables with parameter c=n or truncated heavy-tailed random variables. In both cases, when c goes to infinity or when the truncation is small, the limiting spectrum is a perturbation of the Marchenko-Pastur distribution and we compute its leading term.
In this paper we address the problem of rare-event simulation for heavy-tailed Lévy processes with infinite activities. We propose a strongly efficient importance sampling algorithm that builds upon the sample path large deviations for heavy-tailed Lévy processes, stick-breaking approximation of extrema of Lévy processes, and the randomized debiasing Monte Carlo scheme. The proposed importance sampling algorithm can be applied to a broad class of Lévy processes and exhibits significant improvements in efficiency when compared to crude Monte-Carlo method in our numerical experiments.
Classical Edgeworth expansions provide asymptotic correction terms to the Central Limit Theorem (CLT) up to an order that depends on the number of moments available. In this paper, we provide subsequent correction terms beyond those given by a standard Edgeworth expansion in the general case of regularly varying distributions with diverging moments (beyond the second). The subsequent terms can be expressed in a simple closed form in terms of certain special functions (Dawson's integral and parabolic cylinder functions), and there are qualitative differences depending on whether the number of m…
A telegraph process with an elastic barrier at the origin was studied in [5]; in particular the number of visits of the origin before the absorption is a geometric distributed random variable M. Some asymptotic results (large and moderate deviations) for that model were obtained in [17]. In this paper we study large and moderate deviations for a generalized model where M is a light-tailed distributed random variable.
We study the large-time asymptotic of renewal-reward processes with a heavy-tailed waiting time distribution. It is known that the heavy tail of the distribution produces an extremely slow dynamics, resulting in a singular large deviation function. When the singularity takes place, the bottom of the large deviation function is flattened, manifesting anomalous fluctuations of the renewal-reward processes. In this article, we aim to study how these singularities emerge as the time increases. Using a classical result on the sum of random variables with regularly varying tail, we develop an expans…
A unified treatment for the existence of free energy in several random energy models is presented. If the sequence of distributions associated with the particle systems obeys a large deviation principle, then the free energy exists almost surely. This includes all the known cases as well as some heavy-tailed distributions.
We propose a class of strongly efficient rare event simulation estimators for random walks and compound Poisson processes with a regularly varying increment/jump-size distribution in a general large deviations regime. Our estimator is based on an importance sampling strategy that hinges on the heavy-tailed sample path large deviations result recently established in Rhee, Blanchet, and Zwart (2016). The new estimators are straightforward to implement and can be used to systematically evaluate the probability of a wide range of rare events with bounded relative error. They are "universal" in the…
Establishing a Large Deviation Principle (LDP) proves to be a powerful result for a vast number of stochastic models in many application areas of probability theory. The key object of an LDP is the large deviations rate function, from which probabilistic estimates of rare events can be determined. In order make these results empirically applicable, it would be necessary to estimate the rate function from observations. This is the question we address in this article for the best known and most widely used LDP: Cramér's theorem for random walks. We establish that even when only a narrow LDP hold…
