This paper introduces novel frameworks for large deviations and metastability analysis in heavy-tailed stochastic dynamical systems. We develop and apply these frameworks within the context of stochastic difference equation $X^η_{j+1}(x) = X^η_{j}(x) + ηa\big( X^η_{j}(x)\big) + ησ\big( X^η_{j}(x)\big)Z_{j+1}$ and its variation with truncated dynamics $X^{η|b}_{j+1}(x) = X^{η|b}_{j}( x) + φ_b\big(ηa\big( X^{η|b}_{j}( x)\big) + ησ\big( X^{η|b}_{j}( x)\big) Z_{j+1}\big)$, where $φ_b(x) = (x/\|x\|)\max\{\|x\|, b\}$. The truncation operator $φ_b(\cdot)$ is often introduced as a modulation mechanism in heavy-tailed systems, such as stochastic gradient descent algorithms in deep learning. Thus, it is crucial to successfully analyze both $X^η_{j}(x)$ and $X^{η|b}_{j}(x)$. We establish locally uniform sample-path large deviations for both processes and translate these asymptotics into precise characterizations of the joint distributions of the first exit times and exit locations. Our large deviations asymptotics are sharp enough to rigorously characterize \emph{the catastrophe principle} by establishing the distributional limit of the sample paths conditional on the rare events of interest, thereby revealing the most likely paths through which rare events arise in heavy-tailed dynamical systems. Moreover the resulting limit theorem unveils a discrete hierarchy of phase transitions (i.e., exit times) as the truncation threshold $b$ varies. Together, these developments serve as a heavy-tailed counterpart of the classical Freidlin-Wentzell theory. We also present the corresponding results for continuous-time processes in the appendix.
We consider a general class of finite dimensional deterministic dynamical systems with finitely many local attractors $K^i$ each of which supports a unique ergodic probability measure $P^i$, which includes in particular the class of Morse-Smale systems in any finite dimension. The dynamical system is perturbed by a multiplicative non-Gaussian heavy-tailed Lévy type noise of small amplitude $\varepsilon>0$. Specifically we consider perturbations leading to a Itô, Stratonovich and canonical (Marcus) stochastic differential equation. The respective asymptotic first exit time and location problem …
Consider a sequence of continuous-time Markov chains $(X^{(n)}_t:t\ge 0)$ evolving on a fixed finite state space $V$. Let $I_n$ be the level two large deviations rate functional for $X^{(n)}_t$, as $t\to\infty$. Under a hypothesis on the jump rates, we prove that $I_n$ can be written as $I_n = I^{(0)} \,+\, \sum_{1\le p\le q} (1/θ^{(p)}_n) \, I^{(p)}$ for some rate functionals $I^{(p)}$. The weights $θ^{(p)}_n$ correspond to the time-scales at which the sequence of Markov chains $X^{(n)}_t$ exhibit a metastable behavior, and the zero level sets of the rate functionals $I^{(p)}$ identify the me…
A trajectorial large deviation principle is established in a mean field thermodynamic limit for a multiclass loss network with diminishing rates, which may have several stable equilibria. The large deviation limit is identified as a unique solution to a maxingale problem with a Markov property. The invariant measure of the network process obeys a large deviation principle as well. The network is metastable in that it spends exponentially long periods of time in the neighbourhoods of stable equilibria. A specific case of a two--class network with two stable equilibria and one unstable equilibri…
It is well-known that large deviations of random walks driven by independent and identically distributed heavy-tailed random variables are governed by the so-called principle of one large jump. We note that further subtleties hold for such random walks in the large deviation scale which we call hidden large deviation. We apply this idea in the context of queueing processes with heavy-tailed service times and study approximations of severe congestion times for (buffered) queues. We conclude with simulated examples to verify our results.
It is known that large deviations of sums of subexponential random variables are most likely realised by deviations of a single random variable. In this article we give a detailed picture of how subexponential random variables are distributed when a large deviation of their sum is observed.
We prove large deviation results for sums of heavy-tailed random elements in rather general convex cones being semigroups equipped with a rescaling operation by positive real numbers. In difference to previous results for the cone of convex sets, our technique does not use the embedding of cones in linear spaces. Examples include the cone of convex sets with the Minkowski addition, positive half-line with maximum operation and the family of square integrable functions with arithmetic addition and argument rescaling.
A large deviations principle is established for the joint law of the empirical measure and the flow measure of a renewal Markov process on a finite graph. We do not assume any bound on the arrival times, allowing heavy tailed distributions. In particular, the rate functional is in general degenerate (it has a nontrivial set of zeros) and not strictly convex. These features show a behavior highly different from what one may guess with a heuristic Donsker-Varadhan analysis of the problem.
This paper studies the effect of truncation on the large deviations behavior of the partial sum of a triangular array coming from a truncated power law model. Each row of the triangular array consists of i.i.d. random vectors, whose distribution matches a power law on a ball of radius going to infinity, and outside that it has a light-tailed modification. The random vectors are assumed to be R^d-valued. It turns out that there are two regimes depending on the growth rate of the truncating threshold, so that in one regime, much of the heavy tailedness is retained, while in the other regime, the…
We study large deviation probabilities for a sum of dependent random variables from a heavy-tailed factor model, assuming that the components are regularly varying. We identify conditions where both the factor and the idiosyncratic terms contribute to the behaviour of the tail-probability of the sum. A simple conditional Monte Carlo algorithm is also provided together with a comparison between the simulations and the large deviation approximation. We also study large deviation probabilities for stochastic processes with factor structure. The processes involved are assumed to be Levy processes …
This paper investigates the decay rate of the probability that the row sum of a triangular array of truncated heavy tailed random variables is larger than an integer (k) times the truncating threshold, as both - the number of summands and the threshold go to infinity. The method of attack for this problem is significantly different from the one where k is not an integer, and requires much sharper estimates.
Two models of loss networks, introduced by Gibbens et al. and by Antunes et al., are known to exhibit a mean field limiting regime with several stable equilibria. These models are reexamined in the light of Freidlin and Wentzell's large deviation approach of randomly perturbed dynamical systems. Assuming that some of their results still hold under slightly relaxed conditions, the metastability property is derived for both systems. A Lyapunov function exhibited by Antunes et al. is next identified with the quasipotential associated with a slightly modified, asymptotically reversible, Markovian …
Modern risk modelling approaches deal with vectors of multiple components. The components could be, for example, returns of financial instruments or losses within an insurance portfolio concerning different lines of business. One of the main problems is to decide if there is any type of dependence between the components of the vector and, if so, what type of dependence structure should be used for accurate modelling. We study a class of heavy-tailed multivariate random vectors under a non-parametric shape constraint on the tail decay rate. This class contains, for instance, elliptical distribu…
We consider a nearly-elastic model system with one degree of freedom. In each collision with the "wall", the system can either lose or gain a small amount of energy due to stochastic perturbation. The weak limit of the corresponding slow motion, which is a stochastic process on a graph, is calculated. A large deviation type asymptotics and the metastability of the system are also considered.
In this paper we develop a metastability theory for a class of stochastic reaction-diffusion equations exposed to small multiplicative noise. We consider the case where the unperturbed reaction-diffusion equation features multiple asymptotically stable equilibria. When the system is exposed to small stochastic perturbations, it is likely to stay near one equilibrium for a long period of time, but will eventually transition to the neighborhood of another equilibrium. We are interested in studying the exit time from the full domain of attraction (in a function space) surrounding an equilibrium a…
We prove that the largest and smallest root in modulus of random Kac polynomials have a non-universal behavior. They do not converge towards the edge of the support of the limiting distribution of the zeros. This non-universality is surprising as the large deviation principle for the empirical measure is universal. This is in sharp contrast with random matrix theory where the large deviation principle is non-universal but the fluctuations of the largest eigenvalue are universal. We show that the modulus of the largest zero is heavy tailed, with a number of finite moments bounded from above by …
We consider the FCFS G/G/n queue in the Halfin-Whitt regime, in the presence of heavy-tailed distributions (i.e. infinite variance). We prove that under minimal assumptions, i.e. only that processing times have finite 1 + epsilon moment and inter-arrival times have finite second moment, the sequence of stationary queue length distributions, normalized by $n^{\frac{1}{2}}$, is tight. All previous tightness results for the stationary queue length required that processing times have finite 2 + epsilon moment. Furthermore, we develop simple and explicit bounds on the stationary queue length in tha…
We extend classical results by A. V. Nagaev [Izv. Akad. Nauk UzSSR Ser. Fiz.--Mat. Nauk 6 (1969) 17--22, Theory Probab. Appl. 14 (1969) 51--64, 193--208] on large deviations for sums of i.i.d. regularly varying random variables to partial sum processes of i.i.d. regularly varying vectors. The results are stated in terms of a heavy-tailed large deviation principle on the space of càdlàg functions. We illustrate how these results can be applied to functionals of the partial sum process, including ruin probabilities for multivariate random walks and long strange segments. These results make preci…
We study deviation of U-statistics when samples have heavy-tailed distribution so the kernel of the U-statistic does not have bounded exponential moments at any positive point. We obtain an exponential upper bound for the tail of the U-statistics which clearly denotes two regions of tail decay, the first is a Gaussian decay and the second behaves like the tail of the kernel. For several common U-statistics, we also show the upper bound has the right rate of decay as well as sharp constants by obtaining rough logarithmic limits which in turn can be used to develop LDP for U-statistics. In spite…
We consider a multivariate heavy-tailed stochastic volatility model and analyze the large-sample behavior of its sample covariance matrix. We study the limiting behavior of its entries in the infinite-variance case and derive results for the ordered eigenvalues and corresponding eigenvectors. Essentially, we consider two different cases where the tail behavior either stems from the i.i.d. innovations of the process or from its volatility sequence. In both cases, we make use of a large deviations technique for regularly varying time series to derive multivariate $α$-stable limit distributions o…
We investigate the probability that an insurance portfolio gets ruined within a finite time period under the assumption that the r largest claims are (partly) reinsured. We show that for regularly varying claim sizes the probability of ruin after reinsurance is also regularly varying in terms of the initial capital, and derive an explicit asymptotic expression for the latter. We establish this result by leveraging recent developments on sample-path large deviations for heavy tails. Our results allow, on the asymptotic level, for an explicit comparison between two well-known large-claim reinsur…
We study a zero-range process with system-size dependent jump rates, which is known to exhibit a discontinuous condensation transition. Metastable homogeneous phases and condensed phases coexist in extended phase regions around the transition, which have been fully characterized in the context of the equivalence and non-equivalence of ensembles. In this communication we report rigorous results on the large deviation properties and the free energy landscape which determine the metastable dynamics of the system. Within the condensed phase region we identify a new dynamic transition line which se…
We consider a stochastic fluid network where the external input processes are compound Poisson with heavy-tailed Weibullian jumps. Our results comprise of large deviations estimates for the buffer content process in the vector-valued Skorokhod space which is endowed with the product $J_1$ topology. To illustrate our framework, we provide explicit results for a tandem queue. At the heart of our proof is a recent sample-path large deviations result, and a novel continuity result for the Skorokhod reflection map in the product $J_1$ topology.
We study sample-path large deviations for Lévy processes and random walks with heavy-tailed jump-size distributions that are of Weibull type. Our main results include an extended form of an LDP (large deviations principle) in the $J_1$ topology, and a full LDP in the $M_1'$ topology. The rate function can be represented as the solution to a quasi-variational problem. The sharpness and applicability of these results are illustrated by a counterexample proving the nonexistence of a full LDP in the $J_1$ topology, and by an application to a first passage problem.
We establish a large deviation principle for the occupation measure of the stochastic real Ginzburg-Landau equation driven by $α$-stable noises. The proof is based on a hyper-exponential recurrence criterion. Our result indicates a phenomenon that strong dissipation beats heavy tailed noises to produce a large deviation, it seems to us that this phenomenon has not been reported in the known literatures.
We consider a GI/GI/1 queue with the shortest remaining processing time discipline (SRPT) and light-tailed service times. Our interest is focused on the tail behavior of the sojourn-time distribution. We obtain a general expression for its large-deviations decay rate. The value of this decay rate critically depends on whether there is mass in the endpoint of the service-time distribution or not. An auxiliary priority queue, for which we obtain some new results, plays an important role in our analysis. We apply our SRPT-results to compare SRPT with FIFO from a large-deviations point of view.
We obtain concentration and large deviation for the sums of independent and identically distributed random variables with heavy-tailed distributions. Our concentration results are concerned with random variables whose distributions satisfy $\mathbb{P}(X>t) \leq {\rm e}^{- I(t)}$, where $I: \mathbb{R} \rightarrow \mathbb{R}$ is an increasing function and $I(t)/t \rightarrow α\in [0, \infty)$ as $t \rightarrow \infty$. Our main theorem can not only recover some of the existing results, such as the concentration of the sum of subWeibull random variables, but it can also produce new results for th…
We study asymptotic properties of spatially non-homogeneous random walks with non-integrable increments, including transience, almost-sure bounds, and existence and non-existence of moments for first-passage and last-exit times. In our proofs we also make use of estimates for hitting probabilities and large deviations bounds. Our results are more general than existing results in the literature, which consider only the case of sums of independent (typically, identically distributed) random variables. We do not assume the Markov property. Existing results that we generalize include a circle of i…
In this paper we study the asymptotic decay of finite time ruin probabilities for an insurance company that faces heavy-tailed claims, uses predictable investment strategies and makes investments in risky assets whose prices evolve according to quite general semimartingales. We show that the ruin problem corresponds to determining hitting probabilities for the solution to a randomly perturbed stochastic integral equation. We derive a large deviation result for the hitting probabilities that holds uniformly over a family of semimartingales and show that this result gives the asymptotic decay of…
Large Deviation Upper Bounds and Improved MSE Rates of Nonlinear SGD: Heavy-tailed Noise and Power of Symmetry
Carnegie Mellon University（卡内基梅隆大学）；Faculty of Technical Sciences, University of Novi Sad（诺维萨德大学技术科学学院）；Faculty of Sciences, University of Novi Sad（诺维萨德大学理学院）
We study a large class of reversible Markov chains with discrete state space and transition matrix $P_N$. We define the notion of a set of {\it metastable points} as a subset of the state space $\G_N$ such that (i) this set is reached from any point $x\in \G_N$ without return to x with probability at least $b_N$, while (ii) for any two point x,y in the metastable set, the probability $T^{-1}_{x,y}$ to reach y from x without return to x is smaller than $a_N^{-1}\ll b_N$. Under some additional non-degeneracy assumption, we show that in such a situation: \item{(i)} To each metastable point corres…
In this paper we address the problem of rare-event simulation for heavy-tailed Lévy processes with infinite activities. We propose a strongly efficient importance sampling algorithm that builds upon the sample path large deviations for heavy-tailed Lévy processes, stick-breaking approximation of extrema of Lévy processes, and the randomized debiasing Monte Carlo scheme. The proposed importance sampling algorithm can be applied to a broad class of Lévy processes and exhibits significant improvements in efficiency when compared to crude Monte-Carlo method in our numerical experiments.
Classical Edgeworth expansions provide asymptotic correction terms to the Central Limit Theorem (CLT) up to an order that depends on the number of moments available. In this paper, we provide subsequent correction terms beyond those given by a standard Edgeworth expansion in the general case of regularly varying distributions with diverging moments (beyond the second). The subsequent terms can be expressed in a simple closed form in terms of certain special functions (Dawson's integral and parabolic cylinder functions), and there are qualitative differences depending on whether the number of m…
A telegraph process with an elastic barrier at the origin was studied in [5]; in particular the number of visits of the origin before the absorption is a geometric distributed random variable M. Some asymptotic results (large and moderate deviations) for that model were obtained in [17]. In this paper we study large and moderate deviations for a generalized model where M is a light-tailed distributed random variable.
We study the large-time asymptotic of renewal-reward processes with a heavy-tailed waiting time distribution. It is known that the heavy tail of the distribution produces an extremely slow dynamics, resulting in a singular large deviation function. When the singularity takes place, the bottom of the large deviation function is flattened, manifesting anomalous fluctuations of the renewal-reward processes. In this article, we aim to study how these singularities emerge as the time increases. Using a classical result on the sum of random variables with regularly varying tail, we develop an expans…
