Let $W$ be a random variable with mean zero and variance $σ^2$. The distribution of a variate $W^*$, satisfying $EWf(W)=σ^2 Ef'(W^*)$ for smooth functions $f$, exists uniquely and defines the zero bias transformation on the distribution of $W$. The zero bias transformation shares many interesting properties with the well known size bias transformation for non-negative variables, but is applied to variables taking on both positive and negative values. The transformation can also be defined on more general random objects. The relation between the transformation and the expression $wf'(w)-σ^2 f''(w)$ which appears in the Stein equation characterizing the mean zero, variance $σ^2$ normal $σZ$ can be used to obtain bounds on the difference $E\{h(W/σ)-h(Z)\}$ for smooth functions $h$ by constructing the pair $(W,W^*)$ jointly on the same space. When $W$ is a sum of $n$ not necessarily independent variates, under certain conditions which include a vanishing third moment, bounds on this difference of the order $1/n$ for classes of smooth functions $h$ may be obtained. The technique is illustrated by an application to simple random sampling.
We generalize the well-known zero bias distribution and the $λ$-Stein pair to an approximate zero bias distribution and an approximate $λ,R$-Stein pair, respectively. Berry Esseen type bounds to the normal, based on approximate zero bias couplings and approximate $λ,R$-Stein pairs, are obtained using Stein's method. The bounds are then applied to combinatorial central limit theorems where the random permutation has the Ewens $\mathcal{E}_θ$ distribution with $θ>0$ which can be specialized to the uniform distribution by letting $θ=1$. The family of the Ewens distributions appears in the context…
In this paper we extend Stein's method to the distribution of the product of $n$ independent mean zero normal random variables. A Stein equation is obtained for this class of distributions, which reduces to the classical normal Stein equation in the case $n=1$. This Stein equation motivates a generalisation of the zero bias transformation. We establish properties of this new transformation, and illustrate how they may be used together with the Stein equation to assess distributional distances for statistics that are asymptotically distributed as the product of independent central normal random…
We provide a general theorem bounding the error in the approximation of a random measure of interest--for example, the empirical population measure of types in a Wright-Fisher model--and a Dirichlet process, which is a measure having Poisson-Dirichlet distributed atoms with i.i.d. labels from a diffuse distribution. The implicit metric of the approximation theorem captures the sizes and locations of the masses, and so also yields bounds on the approximation between the masses of the measure of interest and the Poisson-Dirichlet distribution. We apply the result to bound the error in the approx…
We propose in this work RBM-SVGD, a stochastic version of Stein Variational Gradient Descent (SVGD) method for efficiently sampling from a given probability measure and thus useful for Bayesian inference. The method is to apply the Random Batch Method (RBM) for interacting particle systems proposed by Jin et al to the interacting particle systems in SVGD. While keeping the behaviors of SVGD, it reduces the computational cost, especially when the interacting kernel has long range. Numerical examples verify the efficiency of this new version of SVGD.
Stein's method is used to obtain two theorems on multivariate normal approximation. Our main theorem, Theorem 1.2, provides a bound on the distance to normality for any nonnegative random vector. Theorem 1.2 requires multivariate size bias coupling, which we discuss in studying the approximation of distributions of sums of dependent random vectors. In the univariate case, we briefly illustrate this approach for certain sums of nonlinear functions of multivariate normal variables. As a second illustration, we show that the multivariate distribution counting the number of vertices with given deg…
Using Stein's method techniques, we develop a framework which allows one to bound the error terms arising from approximation by the Laplace distribution and apply it to the study of random sums of mean zero random variables. As a corollary, we deduce a Berry-Esseen type theorem for the convergence of certain geometric sums. Our results make use of a second order characterizing equation and a distributional transformation which is related to zero-biasing.
Stein's method is applied to obtain a general Cramer-type moderate deviation result for dependent random variables whose dependence is defined in terms of a Stein identity. A corollary for zero-bias coupling is deduced. The result is also applied to a combinatorial central limit theorem, a general system of binary codes, the anti-voter model on a complete graph, and the Curie-Weiss model. A general moderate deviation result for independent random variables is also proved.
To improve the efficiency of Monte Carlo estimation, practitioners are turning to biased Markov chain Monte Carlo procedures that trade off asymptotic exactness for computational speed. The reasoning is sound: a reduction in variance due to more rapid sampling can outweigh the bias introduced. However, the inexactness creates new challenges for sampler and parameter selection, since standard measures of sample quality like effective sample size do not account for asymptotic bias. To address these challenges, we introduce a new computable quality measure based on Stein's method that quantifies …
A new class of distributional transformations is introduced, characterized by equations relating function weighted expectations of test functions on a given distribution to expectations of the transformed distribution on the test function's higher order derivatives. The class includes the size and zero bias transformations, and when specializing to weighting by polynomial functions, relates distributional families closed under independent addition, and in particular the infinitely divisible distributions, to the family of transformations induced by their associated orthogonal polynomial system…
We derive explicit central moment inequalities for random variables that admit a Stein coupling, such as exchangeable pairs, size--bias couplings or local dependence, among others. The bounds are in terms of moments (not necessarily central) of variables in the Stein coupling, which are typically local in some sense, and therefore easier to bound. In cases where the Stein couplings have the kind of behaviour leading to good normal approximation, the central moments are closely bounded by those of a normal. We show how the bounds can be used to produce concentration inequalities, and compare th…
In this paper, we explore some links between transforms derived by Stein's method and concentration inequalities. In particular, we show that the stochastic domination of the zero bias transform of a random variable is equivalent to sub-Gaussian concentration. For this purpose a new stochastic order is considered. In a second time, we study the case of functions of slightly dependent light-tailed random variables. We are able to recover the famous McDiarmid type of concentration inequality for functions with the bounded difference property. Additionally, we obtain new concentration bounds when…
Distributional transformations characterized by equations relating expectations of test functions weighted by a given biasing function on the original distribution to expectations of the test function's higher derivatives with respect to the transformed distribution play a great role in Stein's method and were, in great generality, first considered by Goldstein and Reinert \cite{GolRei05b}. We prove two abstract existence and uniqueness results for such distributional transformations, generalizing their $X-P$ bias transformation. On the one hand, we show how one can abandon previously necessar…
Berry Esseen type bounds to the normal, based on zero- and size-bias couplings, are derived using Stein's method. The zero biasing bounds are illustrated with an application to combinatorial central limit theorems where the random permutation has either the uniform distribution or one which is constant over permutations with the same cycle type and having no fixed points. The size biasing bounds are applied to the occurrences of fixed relatively ordered sub-sequences (such as rising sequences) in a random permutation, and to the occurrences of patterns, extreme values, and subgraphs on finite …
In this paper, we prove that the weak error between a stochastic differential equation with nonlinearity in the sense of McKean given by moments and its approximation by the Euler discretization with time-step h of a system of N interacting particles is O(1/N + h). We provide numerical experiments confirming this behaviour and showing that it extends to more general mean-field interaction and study the efficiency of the antithetic sampling technique on the same examples.
The geometric sum plays a significant role in risk theory and reliability theory \cite{Kala97} and a prototypical example of the geometric sum is Rényi's theorem~\cite{Renyi56} saying a sequence of suitably parameterised geometric sums converges to the exponential distribution. There is extensive study of the accuracy of exponential distribution approximation to the geometric sum \cite{Sugakova95,Kala97,PekozRollin11} but there is little study on its natural counterpart of gamma distribution approximation to negative binomial sums. In this note, we show that a nonnegative random variable follo…
We describe the distribution of frequencies ordered by sample values in a random sample of size $n$ from the two parameter GEM$(α,θ)$ random discrete distribution on the positive integers. These frequencies are a $($size$-α)$-biased random permutation of the sample frequencies in either ranked order, or in the order of appearance of values in the sampling process. This generalizes a well known identity in distribution due to Donnelly and Tavaré (1986) for $α= 0$ to the case $0 \le α< 1$. This description extends to sampling from Gibbs$(α)$ frequencies obtained by suitable conditioning of the G…
With $X^*$ denoting a random variable with the $X$-size bias distribution, what are all distributions for $X$ such that it is possible to have $X^*=X+Y$, $Y\geq 0$, with $X$ and $Y$ {\em independent}? We give the answer, due to Steutel \cite{steutel}, and also discuss the relations of size biasing to the waiting time paradox, renewal theory, sampling, tightness and uniform integrability, compound Poisson distributions, infinite divisibility, and the lognormal distributions.
