We introduce a new version of Stein's method that reduces a large class of normal approximation problems to variance bounding exercises, thus making a connection between central limit theorems and concentration of measure. Unlike Skorokhod embeddings, the object whose variance must be bounded has an explicit formula that makes it possible to carry out the program more easily. As an application, we derive a general CLT for functions that are obtained as combinations of many local contributions, where the definition of "local" itself depends on the data. Several examples are given, including the solution to a nearest-neighbor CLT problem posed by P. Bickel.
We provide non-asymptotic convergence rates of the Polyak-Ruppert averaged stochastic gradient descent (SGD) to a normal random vector for a class of twice-differentiable test functions. A crucial intermediate step is proving a non-asymptotic martingale central limit theorem (CLT), i.e., establishing the rates of convergence of a multivariate martingale difference sequence to a normal random vector, which might be of independent interest. We obtain the explicit rates for the multivariate martingale CLT using a combination of Stein's method and Lindeberg's argument, which is then used in conjun…
Let $\mathbf{A}=\frac{1}{\sqrt{np}}(\mathbf{X}^T\mathbf{X}-p\mathbf {I}_n)$ where $\mathbf{X}$ is a $p\times n$ matrix, consisting of independent and identically distributed (i.i.d.) real random variables $X_{ij}$ with mean zero and variance one. When $p/n\to\infty$, under fourth moment conditions a central limit theorem (CLT) for linear spectral statistics (LSS) of $\mathbf{A}$ defined by the eigenvalues is established. We also explore its applications in testing whether a population covariance matrix is an identity matrix.
If the log likelihood is approximately quadratic with constant Hessian, then the maximum likelihood estimator (MLE) is approximately normally distributed. No other assumptions are required. We do not need independent and identically distributed data. We do not need the law of large numbers (LLN) or the central limit theorem (CLT). We do not need sample size going to infinity or anything going to infinity. Presented here is a combination of Le Cam style theory involving local asymptotic normality (LAN) and local asymptotic mixed normality (LAMN) and Cramér style theory involving derivatives and…
We analyze the fluctuations of incomplete $U$-statistics over a triangular array of independent random variables. We give criteria for a Central Limit Theorem (CLT, for short) to hold in the sense that we prove that an appropriately scaled and centered version of the U-statistic converges to a normal random variable. Our method of proof relies on a martingale CLT. A possible application -- a CLT for the hitting time for random walk on random graphs -- will be presented in \cite{LoTe20b}
We study the central limit theorem for sums of independent tensor powers, $\frac{1}{\sqrt{d}}\sum\limits_{i=1}^d X_i^{\otimes p}$. We focus on the high-dimensional regime where $X_i \in \mathbb{R}^n$ and $n$ may scale with $d$. Our main result is a proposed threshold for convergence. Specifically, we show that, under some regularity assumption, if $n^{2p-1}\gg d$, then the normalized sum converges to a Gaussian. The results apply, among others, to symmetric uniform log-concave measures and to product measures. This generalizes several results found in the literature. Our main technique is a no…
An upper bound for the Wasserstein distance is provided in the general framework of the Wiener-Poisson space. Is obtained from this bound a second order Poincaré-type inequality which is useful in terms of computations. For completeness sake, is made a survey of these results on the Wiener space, the Poisson space, and the Wiener-Poisson space, and showed several applications to central limit theorems with relevant examples: linear functionals of Gaussian subordinated field (where the subordinated field can be processes like fractional Brownian motion or the solution of the Ornstein-Uhlenbeck …
Linear structural error-in-variables models with univariate observations are revisited for studying modified least squares estimators of the slope and intercept. New marginal central limit theorems (CLT's) are established for these estimators, assuming the existence of four moments for the measurement errors and that the explanatory variables are in the domain of attraction of the normal law. The latter condition for the explanatory variables is used the first time, and is so far the most general in this context. It is also optimal, or nearly optimal, for our CLT's. Moreover, due to the obtain…
Using Bernstein polynomial approximations, we prove the central limit theorem for linear spectral statistics of sample covariance matrices, indexed by a set of functions with continuous fourth order derivatives on an open interval including $[(1-\sqrt{y})^2,(1+\sqrt{y})^2]$, the support of the Marucenko--Pastur law. We also derive the explicit expressions for asymptotic mean and covariance functions.
The (CLT) central limit theorems for generalized Frechet means (data descriptors assuming values in stratified spaces, such as intrinsic means, geodesics, etc.) on manifolds from the literature are only valid if a certain empirical process of Hessians of the Frechet function converges suitably, as in the proof of the prototypical BP-CLT (Bhattacharya and Patrangenaru (2005)). This is not valid in many realistic scenarios and we provide for a new very general CLT. In particular this includes scenarios where, in a suitable chart, the sample mean fluctuates asymptotically at a scale $n^α$ with ex…
We prove a central limit theorem (CLT) for the Frechet mean of independent and identically distributed observations in a compact Riemannian manifold assuming that the population Frechet mean is unique. Previous general CLT results in this setting have assumed that the cut locus of the Frechet mean lies outside the support of the population distribution. So far as we are aware, the CLT in the present paper is the first which allows the cut locus to have co-dimension one or two when it is included in the support of the distribution. A key part of the proof is establishing an asymptotic approxima…
We undertake a precise study of the asymptotic and non-asymptotic properties of stochastic approximation procedures with Polyak-Ruppert averaging for solving a linear system $\bar{A} θ= \bar{b}$. When the matrix $\bar{A}$ is Hurwitz, we prove a central limit theorem (CLT) for the averaged iterates with fixed step size and number of iterations going to infinity. The CLT characterizes the exact asymptotic covariance matrix, which is the sum of the classical Polyak-Ruppert covariance and a correction term that scales with the step size. Under assumptions on the tail of the noise distribution, we …
This paper deals with subspace estimation in the small sample size regime, where the number of samples is comparable in magnitude with the observation dimension. The traditional estimators, mostly based on the sample correlation matrix, are known to perform well as long as the number of available samples is much larger than the observation dimension. However, in the small sample size regime, the performance degrades. Recently, based on random matrix theory results, a new subspace estimator was introduced, which was shown to be consistent in the asymptotic regime where the number of samples and…
This paper deals simultaneously with linear structural and functional error-in-variables models (SEIVM and FEIVM), revisiting in this context generalized and modified least squares estimators of the slope and intercept, and some methods of moments estimators of unknown variances of the measurement errors. New joint central limit theorems (CLT's) are established for these estimators in the SEIVM and FEIVM under some first time, so far the most general, respective conditions on the explanatory variables, and under the existence of four moments of the measurement errors. Moreover, due to them bei…
Consider a random vector $\mathbf{y}=\mathbfΣ^{1/2}\mathbf{x}$, where the $p$ elements of the vector $\mathbf{x}$ are i.i.d. real-valued random variables with zero mean and finite fourth moment, and $\mathbfΣ^{1/2}$ is a deterministic $p\times p$ matrix such that the spectral norm of the population correlation matrix $\mathbf{R}$ of $\mathbf{y}$ is uniformly bounded. In this paper, we find that the log determinant of the sample correlation matrix $\hat{\mathbf{R}}$ based on a sample of size $n$ from the distribution of $\mathbf{y}$ satisfies a CLT (central limit theorem) for $p/n\to γ\in (0, 1…
CLT-Optimal Parameter Error Bounds for Linear System Identification
Ming Hsieh Department of Electrical and Computer Engineering, University of Southern California, Los Angeles, California, USA（明希电气与计算机工程系，南加州大学，洛杉矶，加利福尼亚州，美国）
We formulate and prove a new sufficient conditions for Central Limit Theorem(CLT) in the space of continuous functions in the terms typical for the approximation theory. We prove that the conditions for continuous CLT obtained by N.C.Jain and M.B.Marcus are under some natural additional conditions necessary. We provide also some examples in order to show the exactness of obtained results and illustrate briefly the applications in the Monte-Carlo method.
We provide estimates of the rate of strong approximation and bounds for probabilities of moderate deviations in the CLT for the $L_1$-norm of the kernel density estimator without any assumptions on the density and assuming that the kernel is bounded and has bounded support.
CLT and Edgeworth Expansion for m-out-of-n Bootstrap Estimators of The Studentized Median
Department of Industrial Engineering and Management Science Northwestern University（工业工程与管理科学系西北大学）；Department of Computer Science Northwestern University（计算机科学系西北大学）
In this paper, we establish a high-dimensional CLT for the sample mean of $p$-dimensional spatial data observed over irregularly spaced sampling sites in $\mathbb{R}^d$, allowing the dimension $p$ to be much larger than the sample size $n$. We adopt a stochastic sampling scheme that can generate irregularly spaced sampling sites in a flexible manner and include both pure increasing domain and mixed increasing domain frameworks. To facilitate statistical inference, we develop the spatially dependent wild bootstrap (SDWB) and justify its asymptotic validity in high dimensions by deriving error b…
The law of large numbers (LLN) and central limit theorem (CLT) are long and widely been known as two fundamental results in probability theory. Recently problems of model uncertainties in statistics, measures of risk and superhedging in finance motivated us to introduce, in [4] and [5] (see also [2], [3] and references herein), a new notion of sublinear expectation, called \textquotedblleft% $G$-expectation\textquotedblright, and the related \textquotedblleft$G$-normal distribution\textquotedblright from which we were able to define G-Brownian motion as well as the corresponding stochastic cal…
We combine Stein's method with a version of Malliavin calculus on the Poisson space. As a result, we obtain explicit Berry-Esséen bounds in Central Limit Theorems (CLTs) involving multiple Wiener-Itô integrals with respect to a general Poisson measure. We provide several applications to CLTs related to Ornstein-Uhlenbeck Lévy processes.
We use Stein's method to prove a generalization of the Lindeberg-Feller CLT providing an upper and a lower bound for the superior limit of the Kolmogorov distance between a normally distributed random variable and the rowwise sums of a rowwise independent triangular array of random variables which is asymptotically negligible in the sense of Feller. A natural example shows that the upper bound is of optimal order. The lower bound improves a result by Andrew Barbour and Peter Hall.
We prove abstract bounds on the Wasserstein and Kolmogorov distances between non-randomly centered random sums of real i.i.d. random variables with a finite third moment and the standard normal distribution. Except for the case of mean zero summands, these bounds involve a coupling of the summation index with its size biased distribution as was previously considered in \cite{GolRin96} for the normal approximation of nonnegative random variables. When being specialized to concrete distributions of the summation index like the Binomial, Poisson and Hypergeometric distribution, our bounds turn ou…
We give a stochastic calculus proof of the Central Limit Theorem \[ {\int (L^{x+h}_{t}- L^{x}_{t})^{2} dx- 4ht\over h^{3/2}} \stackrel{\mathcal{L}}{\Longrightarrow}c(\int (L^{x}_{t})^{2} dx)^{1/2} η\] as $h\to 0$ for Brownian local time $L^{x}_{t}$. Here $η$ is an independent normal random variable with mean zero and variance one.
The article is showing sharpness of central limit theorems of Kipnis and Varadhan, Derriennic and Lin, Maxwell and Woodroofe. In the case of the CLT of Derriennic and Lin (for Markov chains with a normal operator) it is shown that the assumption of normality cannot be relaxed. In the case of the CLT of Maxwell and Woodroofe, the example of Peligrad and Utev is improved in the sense of getting a convergence to different laws.
This paper provides a Central Limit Theorem (CLT) for a process $\{θ_n, n\geq 0\}$ satisfying a stochastic approximation (SA) equation of the form $θ_{n+1} = θ_n + γ_{n+1} H(θ_n,X_{n+1})$; a CLT for the associated average sequence is also established. The originality of this paper is to address the case of controlled Markov chain dynamics $\{X_n, n\geq 0 \}$ and the case of multiple targets. The framework also accomodates (randomly) truncated SA algorithms. Sufficient conditions for CLT's to hold are provided as well as comments on how these conditions extend previous works (such as independen…
This paper is concerned with normal approximation under relaxed moment conditions using Stein's method. We obtain the explicit rates of convergence in the central limit theorem for (i) nonlinear statistics with finite absolute moment of order $2+δ\in(2,3];$ (ii) nonlinear statistics with vanishing third moment and finite absolute moment of order $3+δ\in(3,4].$ When applied to specific examples, these rates are of the optimal order $O(n^{-\fracδ{2}})$ and $O(n^{-\frac{1+δ}{2}}).$ Our proof are based on the covariance identify formula and simple observations about the solution of Stein's equatio…
Basel II and Solvency 2 both use the Value-at-Risk (VaR) as the risk measure to compute the Capital Requirements. In practice, to calibrate the VaR, a normal approximation is often chosen for the unknown distribution of the yearly log returns of financial assets. This is usually justified by the use of the Central Limit Theorem (CLT), when assuming aggregation of independent and identically distributed (iid) observations in the portfolio model. Such a choice of modeling, in particular using light tail distributions, has proven during the crisis of 2008/2009 to be an inadequate approximation wh…
In this work, we show that uniform integrability is not a necessary condition for central limit theorems (CLT) to hold for normalized multilevel Monte Carlo (MLMC) estimators and we provide near optimal weaker conditions under which the CLT is achieved. In particular, if the variance decay rate dominates the computational cost rate (i.e., $β> γ$), we prove that the CLT applies to the standard (variance minimizing) MLMC estimator. For other settings where the CLT may not apply to the standard MLMC estimator, we propose an alternative estimator, called the mass-shifted MLMC estimator, to which t…
For $k,m,n\in \mathbb{N}$, we consider $n^k\times n^k$ random matrices of the form $$ \mathcal{M}_{n,m,k}(\mathbf{y})=\sum_{α=1}^mτ_α{Y_α}Y_α^T,\quad Y_α=\mathbf{y}_α^{(1)}\otimes...\otimes\mathbf{y}_α^{(k)}, $$ where $τ_{α}$, $α\in[m]$, are real numbers and $\mathbf{y}_α^{(j)}$, $α\in[m]$, $j\in[k]$, are i.i.d. copies of a normalized isotropic random vector $\mathbf{y}\in \mathbb{R}^n$. For every fixed $k\ge 1$, if the Normalized Counting Measures of $\{τ_{α}\}_α$ converge weakly as $m,n\rightarrow \infty$, $m/n^k\rightarrow c\in \lbrack 0,\infty )$ and $\mathbf{y}$ is a good vector in the se…
Let $\{L^{x}_{t} ; (x,t)\in R^{1}\times R^{1}_{+}\}$ denote the local time of Brownian motion. Our main result is to show that for each fixed $t$ $${\int (L^{x+h}_t- L^x_t)^3 dx-12h\int (L^{x+h}_t - L^x_t)L^x_t dx-24h^{2}t\over h^2} \stackrel{\mathcal{L}}{\Longrightarrow}\sqrt{192}(\int (L^x_t)^3dx)^{1/2}η$$ as $h\to 0$, where $η$ is a normal random variable with mean zero and variance one that is independent of $L^{x}_{t}$. This generalizes our previous result for the second moment. We also explain why our approach will not work for higher moments
Let $X=\{X_{t},t\in R_{+}\}$ be a symmetric Lévy process with local time $\{L^{x}_{t} ; (x,t)\in R^{1}\times R^{1}_{+}\}$. When the Lévy exponent $ψ(\la)$ is regularly varying at infinity with index $1<β\leq 2$ and satisfies some additional regularity conditions && \sqrt{hψ^{2}(1/h)} \lc \int (L^{x+h}_{1}- L^{x}_{1})^{2} dx- E(\int (L^{x+h}_{1}- L^{x}_{1})^{2} dx)\rc\nn && {1 in} \stackrel{\mathcal{L}}{\Longrightarrow} (8c_{β,1})^{1/2} η(\int (L_{1}^{x})^{2} dx)^{1/2} \nn, as $h\rar 0$, where $η$ is a normal random variable with mean zero and variance one that is independent of $L^{x}_{t}$, an…
We introduce higher-order Stein kernels relative to the standard Gaussian measure, which generalize the usual Stein kernels by involving higher-order derivatives of test functions. We relate the associated discrepancies to various metrics on the space of probability measures and prove new functional inequalities involving them. As an application, we obtain new explicit improved rates of convergence in the classical multidimensional CLT under higher moment and regularity assumptions.
Consider throwing $n$ balls at random into $m$ urns, each ball landing in urn $i$ with probability $p_i$. Let $S$ be the resulting number of singletons, i.e., urns containing just one ball. We give an error bound for the Kolmogorov distance from $S$ to the normal, and estimates on its variance. These show that if $n$, $m$ and $(p_i, 1 \leq i \leq m)$ vary in such a way that $\sup_i p_i = O(n^{-1})$, then $S$ satisfies a CLT if and only if $n^2 \sum_i p_i^2$ tends to infinity, and demonstrate an optimal rate of convergence in the CLT in this case. In the uniform case $(p_i \equiv m^{-1}) with $…
Given a graph sequence $\{G_n\}_{n \geq 1}$ denote by $T_3(G_n)$ the number of monochromatic triangles in a uniformly random coloring of the vertices of $G_n$ with $c \geq 2$ colors. This arises as a generalization of the birthday paradox, where $G_n$ corresponds to a friendship network and $T_3(G_n)$ counts the number of triples of friends with matching birthdays. In this paper we prove a central limit theorem (CLT) for $T_3(G_n)$ with explicit error rates. The proof involves constructing a martingale difference sequence by carefully ordering the vertices of $G_n$, based on a certain combinat…
