Large deviation theory has provided important clues for the choice of importance sampling measures for Monte Carlo evaluation of exceedance probabilities. However, Glasserman and Wang [Ann. Appl. Probab. 7 (1997) 731--746] have given examples in which importance sampling measures that are consistent with large deviations can perform much worse than direct Monte Carlo. We address this problem by using certain mixtures of exponentially twisted measures for importance sampling. Their asymptotic optimality is established by using a new class of likelihood ratio martingales and renewal theory.
The seminal papers of Pickands [1,2] paved the way for a systematic study of high exceedance probabilities of both stationary and non-stationary Gaussian processes. Yet, in the vector-valued setting, due to the lack of key tools including Slepian's Lemma, Borell-TIS and Piterbarg inequalities there has not been any methodological development in the literature for the study of extremes of vector-valued Gaussian processes. In this contribution we develop the uniform double-sum method for the vector-valued setting obtaining the exact asymptotics of the exceedance probabilities for both stationary…
Extreme value theory provides an asymptotically justified framework for estimation of exceedance probabilities in regions where few or no observations are available. For multivariate tail estimation, the strength of extremal dependence is crucial and it is typically modeled by a parametric family of spectral distributions. In this work we provide asymptotic bounds on exceedance probabilities that are robust against misspecification of the extremal dependence model. They arise from optimizing the statistic of interest over all dependence models within some neighborhood of the reference model. A…
In a variety of problems in pure and applied probability, it is of relevant to study the large exceedance probabilities of the perpetuity sequence $Y_n := B_1 + A_1 B_2 + \cdots + (A_1 \cdots A_{n-1}) B_n$, where $(A_i,B_i) \subset (0,\infty) \times {\mathbb R}$. Estimates for the stationary tail distribution of $\{ Y_n \}$ have been developed in the seminal papers of Kesten (1973) and Goldie (1991). Specifically, it is well-known that if $M := \sup_n Y_n$, then ${\mathbb P} \left\{ M > u \right\} \sim {\cal C}_M u^{-ξ}$ as $u \to \infty$. While much attention has been focused on extending thi…
We develop importance sampling based efficient simulation techniques for three commonly encountered rare event probabilities associated with random walks having i.i.d. regularly varying increments; namely, 1) the large deviation probabilities, 2) the level crossing probabilities, and 3) the level crossing probabilities within a regenerative cycle. Exponential twisting based state-independent methods, which are effective in efficiently estimating these probabilities for light-tailed increments are not applicable when the increments are heavy-tailed. To address the latter case, more complex and …
Sequential Monte Carlo methods which involve sequential importance sampling and resampling are shown to provide a versatile approach to computing probabilities of rare events. By making use of martingale representations of the sequential Monte Carlo estimators, we show how resampling weights can be chosen to yield logarithmically efficient Monte Carlo estimates of large deviation probabilities for multidimensional Markov random walks.
Improving Importance Sampling estimators for rare event probabilities requires sharp approximations of conditional densities. This is achieved for events E_{n}:=(f(X_{1})+...+f(X_{n}))\inA_{n} where the summands are i.i.d. and E_{n} is a large or moderate deviation event. The approximation of the conditional density of the real r.v's X_{i} 's, for 1\leqi\leqk_{n} with repect to E_{n} on long runs, when k_{n}/n\to1, is handled. The maximal value of k compatible with a given accuracy is discussed; algorithms and simulated results are presented.
Spin-transfer torque magnetoresistive random access memory is a potentially transformative technology in the non-volatile memory market. Its viability depends, in part, on one's ability to predictably induce or prevent switching; however, thermal fluctuations cause small but important errors in both the writing and reading processes. Computing these very small probabilities for magnetic nanodevices using naive Monte Carlo simulations is essentially impossible due to their slow statistical convergence, but variance reduction techniques can offer an effective way to improve their efficiency. Her…
Surrogate-Guided Adaptive Importance Sampling for Failure Probability Estimation
Aerospace Engineering and the Institute of Computational and Data Science (ICDS), Penn State（航空航天工程系与计算数据科学研究所，宾夕法尼亚州立大学）；Department of Statistics, Virginia Tech（统计系，弗吉尼亚理工大学）
Both, Bayes Theorem and the cMPE-Method serve for establishing relations between systems of probabilities. By the cMPE-Method non-conditional probabilities are added, by the DPE-Method, they are subtracted, however, in both versions allowing for the non-linearity of non-disjunctive probabilities. Semantic independence is prerequisite. As compared with the results of semantically homogeneous series of observations, the variety of evidence permits arrival at higher probabilities. The advantage of the Bayesian method lies in allowing for evidence extraneous to the domain covered by the hypothesis…
This paper considers importance sampling for estimation of rare-event probabilities in a specific collection of Markovian jump processes used for e.g. modelling of credit risk. Previous attempts at designing importance sampling algorithms have resulted in poor performance and the main contribution of the paper is the design of efficient importance sampling algorithms using subsolutions. The dynamics of the jump processes causes the corresponding Hamilton-Jacobi equations to have an intricate state-dependence, which makes the design of efficient algorithms difficult. We provide theoretical resu…
Importance sampling is a popular method for efficient computation of various properties of a distribution such as probabilities, expectations, quantiles etc. The output of an importance sampling algorithm can be represented as a weighted empirical measure, where the weights are given by the likelihood ratio between the original distribution and the sampling distribution. In this paper the efficiency of an importance sampling algorithm is studied by means of large deviations for the weighted empirical measure. The main result, which is stated as a Laplace principle for the weighted empirical me…
Efficient Rare-Event Simulation for Random Geometric Graphs via Importance Sampling
School of Mathematics and Statistics, University of New South Wales（新南威尔士大学数学与统计学院）；Data Science and Statistics, Aarhus University（奥胡斯大学数据科学与统计学）；Institute of Stochastics, Ulm University（乌尔姆大学随机过程研究所）；School of Mathematics and Physics, The University of Queensland（昆士兰大学数学与物理学院）
This work introduces and compares approaches for estimating rare-event probabilities related to the number of edges in the random geometric graph on a Poisson point process. In the one-dimensional setting, we derive closed-form expressions for a variety of conditional probabilities related to the number of edges in the random geometric graph and develop conditional Monte Carlo algorithms for estimating rare-event probabilities on this basis. We prove rigorously a reduction in variance when compared to the crude Monte Carlo estimators and illustrate the magnitude of the improvements in a simula…
Importance sampling is a variance reduction technique for efficient estimation of rare-event probabilities by Monte Carlo. In standard importance sampling schemes, the system is simulated using an a priori fixed change of measure suggested by a large deviation lower bound analysis. Recent work, however, has suggested that such schemes do not work well in many situations. In this paper we consider dynamic importance sampling in the setting of uniformly recurrent Markov chains. By ``dynamic'' we mean that in the course of a single simulation, the change of measure can depend on the outcome of th…
This paper introduces a new Importance Sampling scheme, called Adaptive Twisted Importance Sampling, which is adequate for the improved estimation of rare event probabilities in he range of moderate deviations pertaining to the empirical mean of real i.i.d. summands. It is based on a sharp approximation of the density of long runs extracted from a random walk conditioned on its end value.
The objective of this work is to quantify the uncertainty in probability of failure estimates resulting from incomplete knowledge of the probability distributions for the input random variables. We propose a framework that couples the widely used Subset simulation (SuS) with Bayesian/information theoretic multi-model inference. The process starts with data used to infer probability distributions for the model inputs. Often such data sets are small. Multi-model inference is used to assess uncertainty associated with the model-form and parameters of these random variables in the form of model pr…
We discuss importance sampling schemes for the estimation of finite time exit probabilities of small noise diffusions that involve escape from an equilibrium. A factor that complicates the analysis is that rest points are included in the domain of interest. We build importance sampling schemes with provably good performance both pre-asymptotically, that is, for fixed size of the noise, and asymptotically, that is, as the size of the noise goes to zero, and that do not degrade as the time horizon gets large. Simulation studies demonstrate the theoretical results.
In this paper a method based on a Markov chain Monte Carlo (MCMC) algorithm is proposed to compute the probability of a rare event. The conditional distribution of the underlying process given that the rare event occurs has the probability of the rare event as its normalizing constant. Using the MCMC methodology a Markov chain is simulated, with that conditional distribution as its invariant distribution, and information about the normalizing constant is extracted from its trajectory. The algorithm is described in full generality and applied to the problem of computing the probability that a h…
A probability model is underdetermined when there is no rational reason to assign a particular infinitesimal value as the probability of single events. Pruss claims that hyperreal probabilities are underdetermined. The claim is based upon external hyperreal-valued measures. We show that internal hyperfinite measures are not underdetermined. The importance of internality stems from the fact that Robinson's transfer principle only applies to internal entities. We also evaluate the claim that transferless ordered fields (surreals, Levi-Civita field, Laurent series) may have advantages over hyperr…
We present a complete framework for determining the asymptotic (or logarithmic) efficiency of estimators of large deviation probabilities and rate functions based on importance sampling. The framework relies on the idea that importance sampling in that context is fully characterized by the joint large deviations of two random variables: the observable defining the large deviation probability of interest and the likelihood factor (or Radon-Nikodym derivative) connecting the original process and the modified process used in importance sampling. We recover with this framework known results about …
We consider a generalization of the discrete-time Self Healing Umbrella Sampling method, which is an adaptive importance technique useful to sample multimodal target distributions. The importance function is based on the weights (namely the relative probabilities) of disjoint sets which form a partition of the space. These weights are unknown but are learnt on the fly yielding an adaptive algorithm. In the context of computational statistical physics, the logarithm of these weights is, up to a multiplicative constant, the free energy, and the discrete valued function defining the partition is …
